FoundationPose使用说明

1、配置cuda环境如下

dengwei@youdao:~$ nvidia-smi Fri Dec1914:48:022025+---------------------------------------------------------------------------------------+|NVIDIA-SMI535.183.01 Driver Version:550.54.14 CUDA Version:12.4||-----------------------------------------+----------------------+----------------------+|GPU Name Persistence-M|Bus-Id Disp.A|Volatile Uncorr. ECC||Fan Temp Perf Pwr:Usage/Cap|Memory-Usage|GPU-Util Compute M.||||MIG M.||=========================================+======================+======================||0NVIDIA GeForce RTX3060Off|00000000:01:00.0 On|N/A||35% 31C P8 10W / 170W|3MiB / 12288MiB|0% Default||||N/A|+-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+|Processes:||GPU GI CI PID Type Process name GPU Memory||ID ID Usage||=======================================================================================||No running processes found|+---------------------------------------------------------------------------------------+ dengwei@youdao:~$ nvcc -V nvcc: NVIDIA(R)Cuda compiler driver Copyright(c)2005-2021 NVIDIA Corporation Built on Sun_Mar_21_19:15:46_PDT_2021 Cuda compilation tools, release11.3, V11.3.58 Build cuda_11.3.r11.3/compiler.29745058_0 dengwei@youdao:~$ls/usr/local/ bin cuda-11.3 cuda-12.2 games lib protobuf-3.11.4 share sunlogin tensor.tar.gz cuda cuda-12.1 etc includemansbin src TensorRT-10.8.0.43 zed dengwei@youdao:~$ ll /usr/local/cuda lrwxrwxrwx1root root2011月1123:36 /usr/local/cuda ->/usr/local/cuda-11.3/

2、参考博客安装foundationPose

https://blog.csdn.net/zml2973002937/article/details/149842806
下载镜像wenbowen123/foundationpose可能不顺利，可以导出本电脑的镜像，之后导入目标设备中；也可以将本电脑正在运行的容器导出来，导入到目标设备，这样容器不用再编译环境，导入运行即可用。镜像或容器的导出导入参考博客：
https://blog.csdn.net/qq_25735431/article/details/143256770

以下是我用的一些安装命令

# git下载FoundationPose代码cd/home/dengwei/PoseDetect/gitclone https://github.com/NVlabs/FoundationPose.git# docker下载镜像cd~ docker pull wenbowen123/foundationpose&&docker tag wenbowen123/foundationpose foundationpose# 配置docker容器内可打开GUI界面dengwei@youdao:~$ xhost +local:root# 使用镜像启动容器dengwei@youdao:~$ docker run -it --gpus all --name foundationpose\-eDISPLAY=$DISPLAY\--envQT_X11_NO_MITSHM=1\-v /home/dengwei/PoseDetect/FoundationPose:/workspace/FoundationPose\-v /tmp/.X11-unix:/tmp/.X11-unix:rw\-v ~/.Xauthority:/root/.Xauthority:rw\wenbowen123/foundationposebash# 查看正在运行的容器dengwei@youdao:~$ dockerps-a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 2b45e0ae068d wenbowen123/foundationpose"bash"5weeks ago Up5weeks foundationpose# 进入该容器dengwei@youdao:~$ dockerexec-it 2b45e0ae068dbash(my)root@2b45e0ae068d:/##进入容器内目录(my)root@2b45e0ae068d:/# cd /workspace/FoundationPose(my)root@2b45e0ae068d:/workspace/FoundationPose## 构建项目（首次运行需要）(my)root@2b45e0ae068d:/workspace/FoundationPose# bash build_all.sh# 运行 demo(my)root@2b45e0ae068d:/workspace/FoundationPose# python run_demo.py --debug 2#以后不小心退出了容器，只要没删除容器，则可以重启容器：dengwei@youdao:~$ docker restart foundationpose dengwei@youdao:~$ dockerexec-it foundationposebash(my)root@2b45e0ae068d:/# cd /workspace/FoundationPose(my)root@2b45e0ae068d:/workspace/FoundationPose# python run_demo.py --debug 2

3、制作数据集进行目标姿态检测（自备3D模型）

目标物体的3D模型需要依赖iPhone 12 以上并且需要是PRO版本的（有深度相机） 或者是iPad PRO M1以上的设备，用AR Code应用app进行扫描得到，仅扫描模型无需花钱。
参考博客：https://www.cnblogs.com/day1024/p/18948051
已制作好的一份数据所在目录为：

/home/dengwei/PoseDetect/FoundationPose/demo_data/data1

修改文件…/FoundationPose/run_demo.py指定为你自己的数据即可跑出实时检测姿态的结果。

4、Grounded-Segment-Anything 配置与运行

该模型的功能是安装文字提示对图片进行语义分割。
参考博客安装python虚拟环境：https://blog.csdn.net/Acecai01/article/details/148233545

dengwei@youdao:~$ pyenvinstall3.8.10 dengwei@youdao:~$ pyenv virtualenv3.8.10 SAM dengwei@youdao:~$ pyenv activate SAM(SAM)dengwei@youdao:~$cd~/PoseDetect/(SAM)dengwei@youdao:~/PoseDetect$gitclone https://github.com/IDEA-Research/Grounded-Segment-Anything.git(SAM)dengwei@youdao:~/PoseDetect$cdGrounded-Segment-Anything(SAM)dengwei@youdao:~/PoseDetect/Grounded-Segment-Anything$cdGroundingDINO(SAM)dengwei@youdao:~/PoseDetect/Grounded-Segment-Anything/GroundingDINO$ python setup.py build_ext --inplace

可能还需安装其他python模块，下载模型权重，请参考博客配置：
https://www.cnblogs.com/wmt0822/p/19005202
使用模型对图片进行分割：

# 第1步骤，打开梯子，连接外网# 第2步骤，(SAM)dengwei@youdao:~/PoseDetect/Grounded-Segment-Anything$ python grounded_sam_demo.py --config GroundingDINO/groundingdino/config/GroundingDINO_SwinT_OGC.py\--grounded_checkpoint groundingdino_swint_ogc.pth\--sam_checkpoint sam_vit_h_4b8939.pth\--input_image /home/dengwei/PoseDetect/FoundationPose/demo_data/data1/rgb/0000000.png\--output_dir"outputs"\--box_threshold0.3\--text_threshold0.25\--text_prompt"bolt with nut"

注意上述使用命令中需要修改成你自己的图片路径，以及目标名字。此外，如果你用多张图片需要分割，可以让AI参考代码grounded_sam_demo.py写一个处理某个目录内多张图片的代码。