silviazuffi / smalr_online Goto Github PK

hi,@silviazuffi ,
I would like to know how to align SMAL model to new species. Please tell me what I should to do. I'm very confused.

Hello, I have the problem shown in follow, I think I followed by the instructions of README completely. I have no idea how to fix it, could u please give me some hints? BTW, I have got a figure of key points of test image.

1
Scaling images of: 0.75
1
Scaling images of: 0.75
1
Scaling images of: 0.75
1
Scaling images of: 0.75
1
Scaling images of: 0.75
/home/lisicheng/smalr/src/smalr/mycore/io.py:58: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
v_ids = np.array([np.atleast_1d(v) - 1 for v in v_ids])
2.13e+04 | cam: 2.13e+04 | cam_t: 0.00e+00
Traceback (most recent call last):
File "/home/lisicheng/smalr/src/smalr/run_on_animals.py", line 96, in
compute_clips(family, model_name, shape_data_name, base_dirs, save_base_dir,
File "/home/lisicheng/smalr/src/smalr/compute_clips.py", line 326, in compute_clips
bodys = compute_multiple_unsynch_frames_from_annotations(bodys, bodys_imgs, model,
File "/home/lisicheng/smalr/src/smalr/compute_clips.py", line 99, in compute_multiple_unsynch_frames_from_annotations
model, keypoints[ic], key_vids[ic], _, landmarks[ic], landmarks_names[ic] = get_annotation(
File "/home/lisicheng/smalr/src/smalr/track_frame_fit.py", line 77, in get_annotation
rot, trans = estimate_global_pose(landmarks, keypoint_vids, model, cam, img, viz=viz)
File "/home/lisicheng/smalr/src/smalr/estimate_global_pose.py", line 146, in estimate_global_pose
ch.minimize( obj, x0=free_variables, method='dogleg', callback=on_step, options={'maxiter': 100, 'e_3': .0001})
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/chumpy/optimization.py", line 41, in minimize
return minimize_dogleg(fun, free_variables=x0, on_step=callback, **options)
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/chumpy/optimization_internal.py", line 408, in minimize_dogleg
state.updateJ(obj)
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/chumpy/optimization_internal.py", line 304, in updateJ
self.J = obj.J
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/chumpy/optimization_internal.py", line 105, in J
result = self.dr_wrt(self.x, profiler=self.profiler).copy()
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/chumpy/optimization_internal.py", line 50, in dr_wrt
new_jac = self.obj.dr_wrt(freevar, profiler=profiler)
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/chumpy/ch.py", line 772, in dr_wrt
dr2 = p.dr_wrt(wrt, profiler=profiler)
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/chumpy/ch.py", line 772, in dr_wrt
dr2 = p.dr_wrt(wrt, profiler=profiler)
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/chumpy/ch.py", line 772, in dr_wrt
dr2 = p.dr_wrt(wrt, profiler=profiler)
[Previous line repeated 1 more time]
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/chumpy/ch.py", line 775, in dr_wrt
indirect_dr = self.compute_rop(p, rhs=dr2)
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/chumpy/ch.py", line 724, in compute_rop
dr = self._compute_dr_wrt_sliced(wrt)
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/chumpy/ch.py", line 289, in _compute_dr_wrt_sliced
result = self.compute_dr_wrt(wrt)
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/opendr-0.77-py3.8-linux-x86_64.egg/opendr/camera.py", line 104, in compute_dr_wrt
result = sp.csc_matrix((data, (IS, JS)))
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/scipy/sparse/compressed.py", line 54, in init
other = self.class(coo_matrix(arg1, shape=shape,
File "/home/lisicheng/anaconda3/envs/smalr/lib/python3.8/site-packages/scipy/sparse/coo.py", line 147, in init
raise ValueError('cannot infer dimensions from zero '
ValueError: cannot infer dimensions from zero sized index arrays

Process finished with exit code 1

Thanks for releasing your code! Just a small note, load_keymapping() in mycore/io.py, map_path is a hardcoded path and I think should be changed to

map_path = '../../src/annotate_kp_matlab/ferrari2smpl_%s.mat' % animal

Hi, may I ask for how to align SMAL model to a new 3D scan?
We already got the CT high-resolution scan. The code for the original SMAL model is not provided.
Therefore, I tried to fit the scan with the SMAL model based on Flame (https://github.com/Rubikplayer/flame-fitting). I follow the code of FLAME and use a similar objective function ( scan-to-mesh distance, landmarks distance, pose penalty, shape penalty with similar weights). I defined 54 points (from head to tail) on both the SMAL model and our CT scan using MeshLab as FLAME. However, the result of fitting SMAL to a scan based on FLAME code is very poor.
To eliminate the possibility that the SMAL model did not cover rodents, I also try to fit the Bear mesh provided in the result part of this SMALR project. It is also not working.
Could you please give me some advice on it? Is it possible to share the code about how to build a SMAL model based on scan?

Dear Silvia,

I would like to ask you how long the process of generating texture from a single image on a GPU machine took for you. For me, it takes 5-7 minutes and I was wondering if it is normal.

Best regards
Ruslan

Hi, I want to know how to make silhouettes for animal img. Can you give me some advices?

Thank you for the code for python3.x!!

I hope to fit the SMAL model to some animal RGB. Could I fit the model into other animals' (e.g. cows') pictures? How should I do it?

After fitting, could I save *.ply or *.obj files with the parameters (shape and pose) of the fitting results? How to get them?

Hi, I am trying to run the demo code but I got this output and it stuck there like forever:

Iteration 3
shape_w 6000.0
9.49e+07 | cam_t_pos_0: 0.00e+00 | feq_0: 9.47e+02 | fpos_0: 0.00e+00 | kp_proj_0: 5.59e+07 | pose_limit_0: 4.72e+05 | pose_prior_0: 3.52e+07 | pose_prior_tail_0: 3.52e+05 | shape_prior_0: 1.13e+06 | trans_init_0: 1.80e+06
9.48e+07 | cam_t_pos_0: 0.00e+00 | feq_0: 9.44e+02 | fpos_0: 0.00e+00 | kp_proj_0: 5.48e+07 | pose_limit_0: 4.54e+05 | pose_prior_0: 3.58e+07 | pose_prior_tail_0: 3.58e+05 | shape_prior_0: 1.58e+06 | trans_init_0: 1.80e+06
beta here: -0.23, -0.43, -0.01, -0.54, -0.33, 0.14, 0.01, -0.17, 0.03, -0.07, -0.15, 0.08, 0.10, -0.00, -0.28, 0.14, -0.01, 0.20, -0.07, -0.03,
Scale 6
weights: s2m 2000.00 m2s 10000.00
nan | cam_t_pos_0: 0.00e+00 | feq_0: 9.44e+02 | fpos_0: 0.00e+00 | kp_proj_0: 5.48e+07 | m2s_0: nan | pose_limit_0: 4.54e+05 | pose_prior_0: 3.58e+07 | pose_prior_tail_0: 3.58e+05 | s2m_0: 1.88e+06 | shape_prior_0: 1.58e+06 | trans_init_0: 1.80e+06

Would you please give me a hint on what I am struggling of and how to get through it ?
I think I have followed all the instructions given in README.md.