{"id":1173,"date":"2023-03-14T13:59:20","date_gmt":"2023-03-14T20:59:20","guid":{"rendered":"https:\/\/www.bellevuecollege.edu\/xrlab\/?p=1173"},"modified":"2023-09-21T10:27:47","modified_gmt":"2023-09-21T17:27:47","slug":"creating-non-human-characters-from-music-videos-with-ai","status":"publish","type":"post","link":"https:\/\/www.bellevuecollege.edu\/xrlab\/2023\/03\/14\/creating-non-human-characters-from-music-videos-with-ai\/","title":{"rendered":"Creating Non-Human Characters from Music Videos with AI"},"content":{"rendered":"\n<p>Phygital+ is an Al Workspace for visual creators and design teams that combines 20+ Al networks in a NoCode tool. In this experiment we utilize this tool to extract human figures from a music video and create new characters in identical poses.<\/p>\n\n\n\n<p>To accomplish this, we began by extracting the frames we wished to recreate from<a href=\"https:\/\/youtu.be\/xU5ikYwR3QQ\"> this youtube video<\/a>. Next, we used four Phygital+ nodes. One node was for importing the frame, while another utilized ControlNet to generate an animal with the same pose as the person in the frame. We also used a Stable Diffusion node to create the background and a final node to merge the background and animal into a single image.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh4.googleusercontent.com\/ydycY-0t9i3FYhBcAE9ncEyIJC7HThJVQFeLxMdAXbCYKpiwpnvZWOMyE2XD_n436Aw1H4jKbvMQF3OWYlTQVOngH9oDy5aM4h4eKP63BXfaNcNK43FVQB3skiQ35y3IBbNbPXig3XwoQ6vx\" alt=\"Flow chart for Creating Non-Human Characters from Music Videos with AI\"  class=\"img-fluid\" \/><\/figure>\n\n\n\n<p>We repeated this process with all the frames we wanted to recreate, you can see the final result on this video:<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Creating Non-Human Characters from Music Videos with AI\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/f1PDVGiIntU?wmode=transparent&amp;rel=0&amp;feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p>Thanks to Tosolini Productions and Mar\u00eda S\u00e1nchez Isaza for their collaboration on this R&amp;D.<\/p>\n\n\n\n<div class=\"wp-block-mayflower-blocks-row row\">\n<div class=\"wp-block-mayflower-blocks-column col-md\"><\/div>\n\n\n\n<div class=\"wp-block-mayflower-blocks-column col-md\"><\/div>\n\n\n\n<div class=\"wp-block-mayflower-blocks-column col-md\"><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Phygital+ is an Al Workspace for visual creators and design teams that combines 20+ Al networks in a NoCode tool. In this experiment we utilize this tool to extract human figures from a music video and create new characters in identical poses. To accomplish this, we began by extracting the frames we wished to recreate <a class=\"read-more\" href=\"https:\/\/www.bellevuecollege.edu\/xrlab\/2023\/03\/14\/creating-non-human-characters-from-music-videos-with-ai\/\">...more about Creating Non-Human Characters from Music Videos with AI<\/a><\/p>\n","protected":false},"author":3392,"featured_media":1174,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[1],"tags":[],"class_list":["post-1173","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/www.bellevuecollege.edu\/xrlab\/wp-json\/wp\/v2\/posts\/1173","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.bellevuecollege.edu\/xrlab\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bellevuecollege.edu\/xrlab\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bellevuecollege.edu\/xrlab\/wp-json\/wp\/v2\/users\/3392"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bellevuecollege.edu\/xrlab\/wp-json\/wp\/v2\/comments?post=1173"}],"version-history":[{"count":5,"href":"https:\/\/www.bellevuecollege.edu\/xrlab\/wp-json\/wp\/v2\/posts\/1173\/revisions"}],"predecessor-version":[{"id":1245,"href":"https:\/\/www.bellevuecollege.edu\/xrlab\/wp-json\/wp\/v2\/posts\/1173\/revisions\/1245"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bellevuecollege.edu\/xrlab\/wp-json\/wp\/v2\/media\/1174"}],"wp:attachment":[{"href":"https:\/\/www.bellevuecollege.edu\/xrlab\/wp-json\/wp\/v2\/media?parent=1173"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bellevuecollege.edu\/xrlab\/wp-json\/wp\/v2\/categories?post=1173"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bellevuecollege.edu\/xrlab\/wp-json\/wp\/v2\/tags?post=1173"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}