Dense tracking, mapping and scene labeling using a depth camera/Localizacion, mapeo, y etiquetamiento denso de la escena usando una camara de profundidad

We present a system for dense tracking, 3D reconstruction, and object detection of desktop-like environments, using a depth camera; the Kinect sensor. The camera is moved by hand meanwhile its pose is estimated, and a dense model, with evolving color information of the scene, is constructed. Alternatively, the user can couple the object detection module (YOLO: you only look once [1]) for detecting and propagating to the model information of categories of objects commonly found over desktops, like monitors, keyboards, books, cups, and laptops, getting a model with color associated to object categories. The camera pose is estimated using a model-to-frame technique with a coarse-to-fine iterative closest point algorithm (ICP), achieving a drift-free trajectory, robustness to fast camera motion and to variable lighting conditions. Simultaneously, the depth maps are fused into the volumetric structure from the estimated camera poses. For visualizing an explicit representation of the scene, the marching cubes algorithm is employed. The tracking, fusion, marching cubes, and object detection processes were implemented using commodity graphics hardware for improving the performance of the system. We achieve outstanding results in camera pose, high quality of the model's color and geometry, and stability in color from the detection module (robustness to wrong detections) and successful management of multiple instances of the same category. KEYWORDS: Dense reconstruction, camera tracking, depth sensor, volumetric representation, object detection, multiple instance labeling Presentamos un sistema de localizacion con informacion densa, reconstruccion 3D, y deteccion de objetos en ambientes tipo escritorio, usando una camara de profundidad; el sensor Kinect. La camara se mueve manualmente mientras se estima su posicion, y se construye un modelo denso con informacion de color de la escena que se actualiza permanentemente. El usuario puede, alternativamente, acoplar el modulo de deteccion de objetos (YOLO: you only look once [1]) para detectar y propagar al modelo informacion de categorias de objetos comunmente encontrados sobre escritorios, como monitores, teclados, libros, vasos y laptops, obteniendo un modelo con color asociado a la categoria del objeto. La posicion de la camara es estimada usando una tecnica modelo-frame con el algoritmo iterativo de punto mas cercano (ICP, iterative closest point) con resolucion en niveles, logrando una trayectoria libre de deriva, r