个人简历

浙江大学计算机科学与技术学院教授。于2009年获计算机科学与工程博士学位，是同年度香港中文大学工程学院最优博士论文奖以及香港中文大学青年学者论文奖获得者。攻读博士期间曾作为访问学者于UIUC ECE系Image Foundation and Processing Group交流。2009年1月进入瑞士ETH Zurich BIWI Computer Vision Lab进行博士后阶段研究。担任阿里巴巴-浙江大学前沿技术联合研究中心计算机视觉与视频分析实验室主任，主持的“俪知虚拟人”项目荣获阿里巴巴集团学术合作最佳项目奖。
教学与课程
数据结构基础
计算机视觉
视觉目标识别与检索
信息检索与搜索引擎（香港中文大学吕荣聪教授暑期班）
研究与成果
Weakly Supervised Single Shot Multiple Face Reconstruction
3D Face Tracking
Deformable Surface Tracking
3dreconstruction.gif
Smart Video Clipping
Litchi:TTS-Driven Talking Head(CVPR18/NeuraIPS 2018 Alibaba Demo)
Media coverage:
1)阿里虚拟美女“俪知”亮相，克隆人不只出现在科幻电影里？
2)克隆人不只出现在科幻电影里？阿里虚拟美女“俪知”亮相
3)“俪知”交互仿真虚拟人系统在云栖亮相
Video Object Tracking
3D Building Reconstruction by Facade Segmentation
Visual Odometry and Reconstruction
Image Retrieval and Place Recognition
Semantic Video Segmentation
出版著作
Publication
Preprint
[1]Jialiang Zhang,Lixiang Lin,Jianke Zhu*,and Steven C.H.Hoi,“Weakly-Supervised Multi-Face 3D Reconstruction,”arXiv 2101.02000.[PDF][Code:https://github.com/kalyo-zjl/WM3DR]
[2]Tun Zhu,Yao Hu,Daoxin Zhang,Tianran Wang,Xiaolong Jiang,Jianke Zhu*,Jiawei Li,“Horizontal-to-Vertical Video Conversion,”arXiv 2101.04051.[PDF]

研究领域

计算机视觉
机器学习""

近期论文

Journal Paper
[1]Jialiang Zhang,Lixiang Lin,Jianke Zhu*,Yang Li,Yun-chen Chen,Yao Hu,Steven C.H.Hoi,“Attribute-Aware Pedestrian Detection in a Crowd,”IEEE Trans.on Multimedia,2020.[PDF][Code:https://github.com/kalyo-zjl/APD][Rank 1st on Citypersons]
[2]Hantang Liu,Yinghao Xu,Jialiang Zhang,Jianke Zhu*,Yang Li,Steven C.H.Hoi,“DeepFacade:A Deep Learning Approach to Facade Parsing with Symmetric Loss,”IEEE Trans.on Multimedia,2020.[PDF][Code: https://github.com/liuhantang/DeepFacade]
[3]Yang Li,Florian Richter,Jingpei Lu,Emily Funk,Ryan Orosco,Jianke Zhu*,Michael C.Yip,“SuPer:A Surgical Perception Framework for Endoscopic Tissue Manipulation with Surgical Robotics”,In IEEE Robotics and Automation Letters(RA-L)&Int.Conference on Robotics and Automation(ICRA),2020.[PDF][Project Webpage]
[4]Jialiang Zhang,Xiongwei Wu,Steven C.H.Hoi,Jianke Zhu*,Feature agglomeration networks for single stage face detection.Neurocomputing 380:180-189(2020).[PDF]
[5]Xiongwei Wu,Doyen Sahoo,Daoxin Zhang,Jianke Zhu,Steven C.H.Hoi,Single-shot bidirectional pyramid networks for high-quality object detection.Neurocomputing 401:1-9(2020)[PDF]
[6]Dan Ma,Bin Liu,Zhao Kang,Jiayu Zhou,Jianke Zhu,Zenglin Xu,Two birds with one stone:Transforming and generating facial images with iterative GAN.Neurocomputing 396:278-290(2020)[PDF]
[7]Wei Feng,Ruize Han,Qing Guo,Jianke Zhu,and Song Wang,“Dynamic Saliency-Aware Regularization for Correlation Filter based Object Tracking”,IEEE Trans.on Image Processing 28(7):3232-3245(2019).
[8]Qingqun Ning,Zhao Liu,Jianke Zhu*,Mingli Song,Jiajun Bu and Chun Chen,“Noise-aware Co-segmentation with Local and Global Priors”,Neurocomputing 287:221-231(2018)
[9]Wenjie Song,Yang Li,Jianke Zhu*and Chun Chen,“Temporally-Adjusted Correlation Filter-based Tracking”,Neurocomputing,286:121-129(2018)
[10]Qingqun Ning,Jianke Zhu*and Chun Chen,“Very Fast Semantic Image Segmentation Using Hierarchical Dilation and Feature Refining”,Cognitive Computation 10(1):62-72(2018).
[11]Xingyu Gao,Steven C.H.Hoi,Yongdong Zhang,Jianshe Zhou,Ji Wan,Zhenyu Chen,Jintao Li,Jianke Zhu,“Sparse Online Learning of Image Similarity.”ACM TIST 8(5):64:1-64:22(2017).
[12]Qingqun Ning,Jianke Zhu*,Zhiyuan Zhong and Chun Chen,“Scalable Image Retrieval by Sparse Product Quantization”,IEEE Trans.on Multimedia,March(2017).
[13]Jianke Zhu*,Chenxia Wu,Chun Chen and Deng Cai,“Treelets Binary Feature Retrieval for Fast Keypoint Recognition”,IEEE Trans.on Cybernetics,vol.45,issue 10.Oct.2015,pp.2129–2141.
[14]Wenjie Song,Jianke Zhu*,Yang Li,and Chun Chen,“Image Alignment by Online Robust PCA via Stochastic Gradient Descent,IEEE Trans.Circuits Syst.Video Techn.26(7):1241-1250(2016).
[15]Chaoqun Hong,Jianke Zhu,Jun Yu,Jun Cheng,Xuhui Chen,“Realtime and robust object matching with a large number of templates.”Multim.Tools Appl.75(3):1459-1480(2016)
[16]Zhiyuan Zhong,Jianke Zhu*and Steven C.H.Hoi,“Fast Object Retrieval Using Direct Spatial Matching”,IEEE Trans.on Multimedia,vol.17,issue 8.Aug.2015,pp.1391-1397
[17]Zhao Liu,Jianke Zhu*,Jiajun Bu and Chun Chen,“A survey of human pose estimation:The body parts parsing based methods”,Journal of Visual Communication and Image Representation,vol.32.Oct.2015,pp.10-19.
[18]Xinyu Wang,Jianke Zhu*,Zibin Zheng,Wenjie Song,Yuanhong Shen,and Michael R.Lyu,“A Spatial-Temporal QoS Prediction Approach to Time-Aware Web Service Recommendation”,ACM Trans.on Web,2016.
[19]Xinyu Wang,Jianke Zhu*and Yuanhong Shen,“Network-aware QoS prediction for Service Composition Using Geolocation”,IEEE Trans.on Service Computing(TSC),vol.8,issue 4.July-Aug 2015,pp.630–643.
[20]Dayong Wang,Steven C.H.Hoi,Ying He,Jianke Zhu,Tao Mei and Jiebo Luo,“Retrieval-based Face Annotation by Weak Label Regularized Local Coordinate Coding,”IEEE Trans.on Pattern Analysis and Machine Intelligences(TPAMI),vol.36,March 2014,pp.550-563.
[21]Dayong Wang,Steven C.H.Hoi,Ying He and Jianke Zhu,“Mining weakly labeled web facial images for search-based face annotation,”IEEE Trans.on Knowledge and Data Engineering,vol.26.Jan.2014,pp.166-179.
[22]Luming Zhang,Yue Gao,Chaoqun Hong,Yinfu Feng,Jianke Zhu and Deng Cai,'Feature Correlation Hypergraph:Exploiting High-order Potentials for Multimodal Recognition,'IEEE Trans.on Cybernetics,vol.44.Aug.2014,1397-1407.
[23]Zhao Liu,Jianke Zhu*,Jiajun Bu and Chun Chen,“Object Cosegmentation by Nonrigid Mapping”,Neurocomputing,vol.135,2014,pp.107-116.
[24]Chenxia Wu,Jianke Zhu*,Deng Cai,Chun Chen and Jiajun Bu,“Semi-Supervised Bootstrap Nonlinear Sequential Projection Learning Hashing,”IEEE Trans.on Knowledge and Data Engineering,vol.25,Jun.2013,pp.1380-1393.
[25]Chaoqun Hong and Jianke Zhu,“Hypergraph-based multi-example ranking with sparse representation for transductive learning image retrieval”,Neurocomputing,vol.101,Feb.2013,pp.94-103.
[26]Lei Wu,Steven C.H.Hoi,Rong Jin,Jianke Zhu and Nenghai Yu,“Learning Bregman Distance Functions for Semi-supervised Clustering,”IEEE Trans.on Knowledge and Data Engineering,2012.
[27]Hao Wu,Jiajun Bu,Chun Chen,Jianke Zhu,Lijun Zhang,Haifeng Liu,Can Wang and Deng Cai,“Locally discriminative topic modeling.”Pattern Recognition 45(1):617-625(2012).
[28]Lei Wu,Steven C.H.Hoi,Rong Jin,Jianke Zhu and Nenghai Yu,“Distance Metric Learning from Uncertain Side Information for Automated Photo Tagging,”ACM Trans.on Intelligent Systems and Technology,Nov.,2011.
[29]Jianke Zhu,Steven C.H.Hoi,Michael R.Lyu and Shuicheng Yan,“A Semi-supervised Approach to Near Duplicate Keyframe Detection and Retrieval,”ACM Trans.on Multimedia Computing,Communications and Applications(TOMCCAP),Feb.2011.
[30]Hao Ma,Jianke Zhu,Michael R.Lyu and Irwin King,“Bridging the Semantic Gap between Image Contents and Tags,”IEEE Trans.on Multimedia,August 2010.
[31]Jianke Zhu,Michael R.Lyu and Thomas S.Huang,'A Fast 2D Shape Recovery Approach by Fusing Features and Appearance,'IEEE Trans.on Pattern Analysis and Machine Intelligences(TPAMI),vol.31,July 2009,pp.1210-1224.
[32]Steven C.H.Hoi,Rong Jin,Jianke Zhu and Michael R.Lyu,“Semi-Supervised SVM Batch Mode Active Learning with Applications to Image Retrieval,”ACM Trans.on Information Systems(TOIS),vol.27,May 2009.
[33]Zenglin Xu,Kaizhu Huang,Jianke Zhu,Irwin King and Michael R.Lyu,“A Novel Kernel-based Maximum A Posteriori Classification Method,”Neural Networks,vol.22,Sep.2009,pp.977-987.
[34]Jianke Zhu,Steven C.H.Hoi and Michael R.Lyu,'Robust Regularized Kernel Regression,'IEEE Trans.on Systems,Man,and Cybernetics-Part B:Cybernetics,2008.
[35]Jianke Zhu,Steven C.H.Hoi and Michael R.Lyu,'Face Annotation by Transductive Kernel Fisher Discriminant,'IEEE Trans.on Multimedia,vol.10,Jan.2008,pp.86-96.
Conference Paper
[36]Shu-Ting Shi,Wenhao Zheng,Jun Tang,Qing-Guo Chen,Yao Hu,Jianke Zhu,Ming Li,“Deep Time-Stream Framework for Click-Through Rate Prediction by Tracking Interest Evolution”,AAAI 2020.
[37]Shengyu Zhang,Tan Jiang,Tan Wang,Kun Kuang,Zhou Zhao,Jianke Zhu,Jin Yu,Hongxia Yang,Fei Wu,“DeVLBert:Learning Deconfounded Visio-Linguistic Representations”.ACM Multimedia 2020:4373-4382.
[38]Jianbo Wang,Kai Qiu,Houwen Peng,Jianlong Fu,Jianke Zhu,“AI Coach:Deep Human Pose Estimation and Analysis for Personalized Athletic Training Assistance”,ACM Multimedia 2019:374-382.Full paper in Main Track.
[39]Jianbo Wang,Kai Qiu,Houwen Peng,Jianlong Fu,Jianke Zhu,“AI Coach:Deep Human Pose Estimation and Analysis for Personalized Athletic Training Assistance”,ACM Multimedia 2019:2228-2230.Demo Paper.
[40]Yang Li,Jianke Zhu*,Steven Hoi,Wenjie Song,Zhefeng Wang and Hantang Liu,“Robust Estimation of Similarity Transformation for Visual Object Tracking”,AAAI 2019:8666-8673.[PDF][Code: https://github.com/ihpdep/LDES]
[41]Jianke Zhu,“Image Gradient-based Joint Direct Visual Odometry for Stereo Camera,”IJCAI 2017.
[42]Yang Li,Zhan Xu,Jianke Zhu*,“CFNN:Correlation Filter Neural Network for Visual Object Tracking,”IJCAI 2017.[Code: https://github.com/zhan-xu/CFNN]
[43]Hantang Liu,Jialiang Zhang,Jianke Zhu*,Steven C.H.Hoi,“DeepFacade:A Deep Learning Approach to Facade Parsing,”IJCAI 2017.[Code: https://github.com/liuhantang/DeepFacade]
[44]Yang Li,Jianke Zhu and Steven C.H.Hoi“Reliable Patch Trackers:Robust Visual Tracking by Exploiting Reliable Patches”,CVPR 2015.[Code: https://github.com/ihpdep/rpt]
[45]Yang Li and Jianke Zhu,“A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration”,ECCV 2014 Workshop:VOT’14 Challenge.Ranked 2/38.[Code:https://github.com/ihpdep/samf]
[46]Ji Wan,Dayong Wang,Steven C.H.Hoi,Pengcheng Wu,Jianke Zhu,Yongdong Zhang and Jintao Li,“Deep Learning for Content-based Image Retrieval:A Comprehensive Study”,ACM Multimedia 2104.
[47]Chenxia Wu,Haiqin Yang,Jianke Zhu,Jiemi Zhang,Irwin King and Michael Lyu,“Sparse Poisson Coding for High Dimensional Document Clustering”,IEEE Big Data 2013,short paper.
[48]Yuanhong Shen,Jianke Zhu,Xinyu Wang,Liang Cai,Xiaohu Yang,and Bo Zhou,“Geographic Location-Based Network-aware QoS Prediction for Service Composition”,ICWS 2013,Research Track.
[49]Dayong Wang,Steven C.H.Hoi,Pengcheng Wu,Jianke Zhu,Ying He and Chunyan Miao,“Learning to Name Faces:A Multimodal Learning Scheme for Search-Based Face Annotation”,SIGIR 2013.
[50]Steven C.H.Hoi,Dayong Wang,I Yeu Cheng,Elmer Lin,Jianke Zhu,Ying He and Chunyan Miao,“FANS:Face Annotation by Searching Large-scale Web Facial Images”,WWW 2013,Demo Track.
[51]Jiemi Zhang,Chenxia Wu,Deng Cai and Jianke Zhu,“Bilevel Visual Words Coding for Image Classification”,IJCAI 2013.
[52]Chenxia Wu,Jianke Zhu,Jiemi Zhang,Chun Chen and Deng Cai,“A Convolutional Treelets Binary Feature Approach to Fast Keypoint Recognition,”ECCV 2012.
[53]Chaoqun Hong,Jianke Zhu,Mingli Song and Yinting Wang,“Realtime Object Matching with Robust Dominant Orientation Templates,”ICPR 2012.
[54]Chenxia Wu,Jianke Zhu and Jiemi Zhang,“A Content-based Video Copy Detection Method with Randomly Projected Binary Features,”CVPR Workshop on LSVSM 2012.
[55]Jiajun Bu,Bin Xu,Chenxia Wu,Chun Chen,Jianke Zhu,Deng Cai and Xiaofei He,“Unsupervised face-name association via commute distance”,ACM Multimedia 2012.
[56]Jianke Zhu,Hao Ma,Jiajun Bu and Chun Chen,“Social Recommendation Using Low-Rank Semidefinite Program,”AAAI 2011.
[57]Dayong Wang,Steven C.H.Hoi,Ying He and Jianke Zhu,“Retrieval-based face annotation by weak label regularized local coordinate coding,”ACM Multimedia 2011:353-362.
[58]Jianke Zhu,Luc Van Gool and Steven C.H.Hoi,“Unsupervised Face Alignment by Robust Nonrigid Mapping,”IEEE International Conference on Computer Vision(ICCV2009).
[59]Zenglin Xu,Rong Jin,Jianke Zhu,Irwin King,Michael R.Lyu and Zhirong Yang,'Adaptive Regularization for Transductive Support Vector Machine,'Advances in Neural Information Processing Systems(NIPS2009).
[60]Lei Wu,Rong Jin,Steven C.H.Hoi,Jianke Zhu and Nenghai Yu,'Learning Bregman Distance Functions and Its Application for Semi-Supervised Clustering,'Advances in Neural Information Processing Systems(NIPS2009).
[61]Jianke Zhu,Steven C.H.Hoi and Michael R.Lyu,“Nonrigid Shape Recovery by Gaussian Process Regression,”IEEE Conference on Computer Vision and Pattern Recognition(CVPR2009).
[62]Lei Wu,Steven C.H.Hoi,Rong Jin,Jianke Zhu and Nenghai Yu,“Distance Metric Learning from Uncertain Side Information with Application to Automated Photo Tagging,”ACM Multimedia 2009.
[63]Jianke Zhu,Steven C.H.Hoi,Michael R.Lyu and Shuicheng Yan,“Near-Duplicate Keyframe Retrieval by Nonrigid Image Matching,”ACM Multimedia 2008.
[64]Jianke Zhu,Steven C.H.Hoi,Zenglin Xu and Michael R.Lyu,“An Effective Approach to 3D Deformable Surface Tracking,”The 10th European Conference on Computer Vision(ECCV2008).
[65]Steven C.H.Hoi,Rong Jin,Jianke Zhu and Michael R.Lyu,“Semi-Supervised SVM Batch Mode Active Learning for Image Retrieval,”IEEE Conference on Computer Vision and Pattern Recognition(CVPR2008).
[66]Jianke Zhu and Michael R.Lyu,'Progressive Finite Newton Approach to Real-time Nonrigid Surface Detection,'IEEE Conference on Computer Vision and Pattern Recognition(CVPR2007),June 19-21,2007.
[67]Jianke Zhu,Steven C.H.Hoi and Michael R.Lyu,'A Multi-Scale Tikhonov Regularization Scheme for Implicit Surface Modelling,'IEEE Conference on Computer Vision and Pattern Recognition(CVPR2007),June 19-21,2007.
[68]Zenglin Xu,Rong Jin,Jianke Zhu,Irwin King and Michael R.Lyu,'Efficient Convex Relaxation for Transductive Support Vector Machine,'Advances in Neural Information Processing Systems(NIPS2007).
[69]Jianke Zhu,Steven C.H.Hoi and Michael R.Lyu,“The Real-Time Non-Rigid Shape Recovery via Active Appearance Models for Augmented Reality,”in Proceedings of The 9th European Conference on Computer Vision(ECCV2006),LNCS 3951,Graz,May 7-13,2006,pp.12-23.
[70]Steven C.H.Hoi,Rong Jin,Jianke Zhu and Michael R.Lyu,“Batch Mode Active Learning and Its Application to Medical Image Classification,”The 23th International Conference on Machine Learning(ICML2006),Pittsburgh,June 25-29,2006.
[71]Zenglin Xu,Kaizhu Huang,Jianke Zhu,Irwin King and Michael R.Lyu,'Kernel Maximum a Posteriori Classification with Error Bound Analysis,'International Conference on Neural Information Processing(ICONIP2007),LNCS 4984,2008,pp.841–850.
[72]Zenglin Xu,Jianke Zhu,Michael R.Lyu and Irwin King,'Semi-supervised Spectral Kernel Learning,'International Joint Conference on Neural Networks(IJCNN2007).
[73]Hongbo Deng,Jianke Zhu,Michael R.Lyu,Irwin King,'Two-Stage Multi-Class AdaBoost for Facial Expression Recognition,'International Joint Conference on Neural Networks(IJCNN2007).
[74]Chon Fong Wong,Jianke Zhu,Mang I Vai and Peng Un Mak,“Face Image Retrieval with Relevance Feedback Using Lifting Wavelets Features,”in Applied and Numerical Harmonic Analysis Series,Tao Qian(ed.),Springer,2006.
[75]Jianke Zhu,Steven C.H.Hoi,Edward Yau and Michael R.Lyu,“Automatic 3D Face Modeling Using 2D Active Appearance Models,”in Proceedings of the 13th Pacific Conference on Computer Graphics and Applications(PG2005),Macau,China,October 12-14,2005.
[76]Steven C.H.Hoi,Jianke Zhu and Michael R.Lyu,“CUHK at ImageCLEF 2005:Cross-Language and Cross-Media Image Retrieval,”in Proceedings of Cross Language Evaluation Forum(CLEF2005),LNCS 4022,Vienna,Austria,2006.
[77]Chon Fong Wong,Jianke Zhu,Mang I Vai and Peng Un Mak,“Face Image Retrieval in Video Sequences Using Lifting Wavelets Transform Feature Extraction,”in Proceedings International Symposium on Computer Electronics(ICSE2005),Macau.
[78]Jianke Zhu,Mang I Vai and Peng Un Mak,“Gabor Wavelets Transform and Extended Nearest Feature Space Classifier for Face Recognition,”in Proceedings Third International Conference on Image and Graphics(ICIG2004),Hong Kong,Dec.28-30,2004,pp.372-379.
[79]Jianke Zhu,Mang I Vai and Peng Un Mak,“A New Enhanced Nearest Feature Space(ENFS)Classifier for Gabor Wavelets Features Based Face Recognition,”in Proceedings First International Conference on Biometrics Authentication(ICBA2004),LNCS 3072,Hong Kong,July 15-17,2004,pp.124-131.
[80]Jianke Zhu,Mang I Vai and Peng Un Mak,“Face Recognition Using 2D DCT with PCA.”The 4th Chinese Conference on Biometric Recognition(Sinobiometrics'03),Beijing,PR China.
Thesis
1.Jianke Zhu,PhD Thesis,Supervisor:Prof.Michael R.Lyu,“Deformable Surface Recovery and Its Applications”,Department of Computer Science and Engineering,The Chinese University of Hong Kong,Hong Kong,December,2008.
2.Jianke Zhu,Master Thesis,Supervisor:Prof.Mang I Vai and Prof.Peng Un Mak,“Real-time Face Recognition System”,Department of Electrical and Electronics Engineering,University of Macau,Macau,March,2005.
IEEE资深会员，目前担任国际期刊Neurocomputing和Springer Big Data Analytics编委。是包括 IEEE TPAMI/TIP/TNNLS、ACM Computing Survey/TOIS/TIST在内的多个国际期刊审稿人，以及AAAI/IJCAI/CVPR等会议的PC和SPC。