3D reconstruction is an important technique in the environmental perception and rehabilitation process. With the help of active depth-aware sensors, such as Kinect from Microsoft and SwissRanger, the depth map can be captured at the video frame rate together with color information to enable the real-time reconstruction. Particularly, it features prominently in the activity recognition and remote rehabilitation. Unfortunately, the coarseness of the depth map make it difficult to extract the detailed information in 3D reconstruction of the scene and tracking of thin objects. Especially, geometric distortions occur around the edge of an object. Therefore, this paper presents a confidence weighted real-time depth filter for the edge recovery to reduce the extra artifacts due to the uncertainty of each depth measurement. Also the intensity of depth map is taken into account to optimize the weighting term in the algorithm. Moreover, the GPU implementation guarantees the high computational efficiency for the real-time applications. Experimental results are shown to illustrate the performance of the proposed method by the comparisons with the traditional methods.