Repository navigation
[Question]: Question about camera intrinsic parameters in http_realworld_server.py in StreamVLN #355
Description
Activity
not used
- The camera intrinsic parameters in the training data are consistent with the camera settings used during Habitat evaluation.
- The parts of StreamVLN that utilize camera intrinsic parameters are only used in the ablation study of voxel-based spatial pruning, while DualVLN (InternNav) does not make use of this component.
@yuqiang-yang @fengdelin
Thank you so much for your kind reply, which greatly helps me understand the applicable scenario of this camera intrinsic matrix.
I have followed the intrinsic calculation logic implemented in your project as below to verify the hardcoded intrinsic values:
fx = (width / 2.0) / np.tan(np.deg2rad(fov / 2.0))
fy = fx
cx = (width - 1.0) / 2.0
cy = (height - 1.0) / 2.0I substituted my Habitat simulator RGB sensor configuration (width=640, height=480, hfov=79) into this formula for calculation. The result I obtained is fx ≈ 388.19 and cx = 319.5, which has a huge discrepancy with the fixed values fx=192, cx=191.42857143 written in http_real_world_server.py.
Could you please share the exact sensor resolution and HFOV setting that this hardcoded intrinsic matrix corresponds to? Besides, could you explain why the center coordinate is 191.42857143 instead of the standard theoretical value?
Thank you very much for your time and explanation. I really appreciate your work.In my understanding, our input does not include the intrinsic matrix, so the intrinsic matrix used in deployment may be problematic, although this did not affect the results at the time. The intrinsic matrix in Habitat may need to be checked with reference to Habitat’s coordinate system. I remember that Habitat’s coordinate system convention is different from the common one.
Question
Dear StreamVLN authors,
Thank you very much for your excellent work and open-sourcing the StreamVLN project! It is really helpful for my research on vision-language navigation.
I have a small question about the camera intrinsic parameters defined in http_realworld_server.py (lines 166-169):
"camera_intrinsic": np.array([[192. , 0. , 191.42857143, 0. ],
[ 0. , 192. , 191.42857143, 0. ],
[ 0. , 0. , 1. , 0. ],
[ 0. , 0. , 0. , 1. ]]),
Could you please clarify the origin of these values? Specifically:
Are these values based on a 384×384 resolution with 90° HFOV in Habitat?
Could you please explain how this camera intrinsic matrix was derived?
Could you please clarify whether this camera intrinsic matrix refers to the configuration of the RGB sensor in the Habitat-sim simulator, or the camera intrinsic parameters of the Intel RealSense D400 series camera used for real-world deployment?