{"id":6443,"date":"2023-03-08T11:09:27","date_gmt":"2023-03-08T03:09:27","guid":{"rendered":"https:\/\/www.hh-ri.com\/?p=6443"},"modified":"2023-03-08T11:09:27","modified_gmt":"2023-03-08T03:09:27","slug":"deep-multimodal-learning-for-computer-vision","status":"publish","type":"post","link":"https:\/\/www.hh-ri.com\/2023\/03\/08\/deep-multimodal-learning-for-computer-vision\/","title":{"rendered":"Deep Multimodal Learning for Computer Vision"},"content":{"rendered":"\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"ast-oembed-container \" style=\"height: 100%;\"><iframe title=\"06 Deep Multimodal Learning for Computer Vision\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/ke2okfoqx1c?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/div>\n<\/div><\/figure>\n\n\n\n<p>NExT Forum\uff1a\u591a\u6a21\u614b\u57fa\u790e\u6a21\u578b <\/p>\n\n\n\n<p>\u3010\u8b1b\u8005\u3011 \u8cf4\u5c1a\u5b8f\u526f\u9662\u9577 | \u570b\u7acb\u6e05\u83ef\u5927\u5b78\u96fb\u6a5f\u8cc7\u8a0a\u5b78\u9662 Dr. Shang-Hong Lai, Associate Dean, College of Electrical Engineering and Computer Science, National Tsing Hua University <\/p>\n\n\n\n<p>\u3010\u8b1b\u984c\u3011Deep Multimodal Learning for Computer Vision <\/p>\n\n\n\n<p>\u3010\u6458\u8981\u3011 Human senses the world through multimodal inputs, such as vision, audio, text, haptics, etc. Each modality provides description for objects or scenes based on its unique characteristic representation. Deep multimodal learning has attracted increasing attention since it can benefit model learning from data of different modalities to boost the accuracy and robustness of the deep neural network system. In this talk, I will present some examples of computer vision applications that employ deep multimodal learning. For the first part, I will introduce the multimodal learning approaches for training face recognition systems to enhance the recognition accuracy. In the second part, I will present the text-image co-training approach that has been used for developing accurate image captioning and intelligent document understanding. Finally, I will briefly describe the recent action recognition research works that take advantage of the pre-trained visual-language model to achieve zero-shot learning for spatio-temporal action detection. <\/p>\n\n\n\n<p>\u3010\u8b70\u7a0b\u3011<a href=\"https:\/\/forum.hh-ri.com\/20230303\" target=\"_blank\" rel=\"noreferrer noopener\"> https:\/\/forum.hh-ri.com\/20230303 <\/a><\/p>\n\n\n\n<p>\u4e3b\u8fa6\u55ae\u4f4d\uff1a\u9d3b\u6d77\u7814\u7a76\u9662 \u5354\u8fa6\u55ae\u4f4d\uff1a\u8ca1\u5718\u6cd5\u4eba\u4eba\u5de5\u667a\u6167\u79d1\u6280\u57fa\u91d1\u6703\u3001\u570b\u7acb\u81fa\u7063\u5927\u5b78\u4eba\u5de5\u667a\u6167\u6280\u8853\u66a8\u5168\u5e45\u5065\u5eb7\u7167\u8b77\u806f\u5408\u7814\u7a76\u4e2d\u5fc3<\/p>\n","protected":false},"excerpt":{"rendered":"<p>NExT Forum\uff1a\u591a\u6a21\u614b\u57fa\u790e\u6a21\u578b \u3010\u8b1b\u8005\u3011 \u8cf4\u5c1a\u5b8f\u526f\u9662\u9577 | \u570b\u7acb\u6e05\u83ef\u5927\u5b78\u96fb\u6a5f\u8cc7\u8a0a\u5b78\u9662 Dr. Shang [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":6444,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"disabled","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"footnotes":""},"categories":[57],"tags":[27],"_links":{"self":[{"href":"https:\/\/www.hh-ri.com\/wp-json\/wp\/v2\/posts\/6443"}],"collection":[{"href":"https:\/\/www.hh-ri.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.hh-ri.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.hh-ri.com\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.hh-ri.com\/wp-json\/wp\/v2\/comments?post=6443"}],"version-history":[{"count":1,"href":"https:\/\/www.hh-ri.com\/wp-json\/wp\/v2\/posts\/6443\/revisions"}],"predecessor-version":[{"id":6445,"href":"https:\/\/www.hh-ri.com\/wp-json\/wp\/v2\/posts\/6443\/revisions\/6445"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.hh-ri.com\/wp-json\/wp\/v2\/media\/6444"}],"wp:attachment":[{"href":"https:\/\/www.hh-ri.com\/wp-json\/wp\/v2\/media?parent=6443"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.hh-ri.com\/wp-json\/wp\/v2\/categories?post=6443"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.hh-ri.com\/wp-json\/wp\/v2\/tags?post=6443"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}