MetaMap: Supporting Visual Metaphor Ideation through Multi-dimensional Example-based Exploration

Youwen Kang*, Zhida Sun*, Sitong Wang, Zeyu Huang, Ziming Wu, Xiaojuan Ma.
* contributed equally
The Hong Kong University of Science and Technology

Paper Code Preview Presentation

MetaMap is a design tool which supports amateur designers in the visual metaphor ideation process. We incorporate in MetaMap a 3-dimensional recommendation framework (i.e., semantics, color, shape) in a mind-map structure to aid ideation with diverse exemplar exploration and historical thinking path tracking.

Abstract

Visual metaphors, which are widely used in graphic design, can deliver messages in creative ways by fusing different objects. The keys to creating visual metaphors are diverse exploration and creative combinations, which is challenging with conventional methods like image searching. To streamline this ideation process, we propose to use a mind-map-like structure to recommend and assist users to explore materials. We present MetaMap, a supporting tool which inspires visual metaphor ideation through multi-dimensional example-based exploration. To facilitate the divergence and convergence of the ideation process, MetaMap provides 1) sample images based on keyword association and color filtering; 2) example-based exploration in semantics, color, and shape dimensions; and 3) thinking path tracking and idea recording. We conduct a within-subject study with 24 design enthusiasts by taking a Pinterest-like interface as the baseline. Our evaluation results suggest that MetaMap provides an engaging ideation process and helps participants create diverse and creative ideas.

The system pipeline of MetaMap

siggraphasia16_teaser

To facilitate divergence and convergence activities in the ideation process, MetaMap provides 1) assembly of image search results based on automatic keyword association and color theme filtering; 2) exploration of examples related to a seed image from semantics, color, and shape dimensions; and 3) support for thinking paths tracking and ideas recording. The conventional image searching process is supported by word association to inspire users with more concepts. After identifying an image of interest, users can further explore the recommendation exemplars based on 3-dimensional features: semantics, color, and shape. Users can save useful images and type comments to record their thoughts before a new brainstorming iteration. All the saved mind maps and images record the thinking path generated by users for a quick recollection.

Sample results from our experiment

chi_21_detail

Participant's notes

  • Sewing machine, face mask, worker, table
  • Create disguise, numb life, and never get happiness. That is not the true meaning of life.
  • This is a scene similar to the work of factory workers (mainly want to show that there are many such people). Workers do not have "faces" or "expressions". They are all working overtime to produce their own "masks" and to produce themselves in adult society. The "face" needed. They are not really happy, but treat life as a mechanized and rigid activity. I hope this will arouse everyone's attention to the pursuit of life's interest and true happiness.
chi_21_detail

Participant's notes

  • Cigarette, smoke, people, clock
  • Cigarette is now destroying people's health.
  • The clock represents time, the cigarette is used as a pointer, and the smoke is transformed into numbers 1-12 and expressed in the shape of a person. It means that cigarettes can destroy people's health and chronically destroy lives. The main colors are red, yellow and white, which are the three basic colors for lighting cigarettes. The style of painting is silhouette painting, which conveys the concept in a cartoonish form.
chi_21_detail

Participant's notes

  • Fat Ironman, one bottle of cola
  • Fizzy drinks can destroy a man, even if he is iron Man.
  • The subject of the picture is the exaggerated iron Man. He became fat because he drank too much coke. The deterioration of his function caused his body to leak oil like coke. The most impressive shot of Iron Man is when he kneeled and snapped his fingers, but now he's only holding a bottle of coke which makes him both love and hate.
chi_21_detail

Participant's notes

  • White house roof, people, umbrella
  • Policy is the people's umbrella
  • Government policy on the regulation of medical resources affects the health of the people at the grassroots level and is a protection umbrella for the people
chi_21_detail

Participant's notes

  • 1，2，3，ukulele
  • One, two, three，a quick start for music
  • Using symbol 123 to construct a instrument
chi_21_detail

Participant's notes

  • Candy packaging bag, candy
  • Music is like candy, making people happy and sweet
  • The main body of the picture is a packaging bag of some candies, but unlike what we usually see, the packaging bag in the picture is in the shape of some musical instruments; beside the packaging bag, some candies are scattered. The candies are in the shape of musical notes, symbolizing that music is like candy, people still can enjoy sweetness and happiness in a bitter life.
chi_21_detail

Participant's notes

  • Chopping board, meat, vegetable garnish, meat stick, chef's hand
  • Not only professional musicians, everyone has the ability to music
  • The picture shows a chef holding a meat stick and hitting the meat and making a sound wave on the cutting board. The shape of the sound wave conveys the feeling of sound. The circular cutting board is like a drumhead, and the meat stick is like a drum stick. With the ability to appreciate music, we can experience the happiness no matter we have received professional music theory education or not. This initiates that music brings people happiness in our work and life.
chi_21_detail

Participant's notes

  • Electrical wires, birds
  • Music come from the nature is beautiful.
  • This poster uses five parallel wires to represent the staff and add some birds on the wires to represent the musical notes. We know what birds can sing and the whole poster looks like a music score, so it is not hard to know about the overall message I wanna convey.

BibTeX


                            @inproceedings{10.1145/3411764.3445325,
                            author = {Kang, Youwen and Sun, Zhida and Wang, Sitong and Huang, Zeyu and Wu, Ziming and Ma, Xiaojuan},
                            title = {MetaMap: Supporting Visual Metaphor Ideation through Multi-dimensional Example-based Exploration},
                            year = {2021},
                            isbn = {9781450380966},
                            publisher = {Association for Computing Machinery},
                            address = {New York, NY, USA},
                            url = {https://doi.org/10.1145/3411764.3445325},
                            doi = {10.1145/3411764.3445325},
                            booktitle = {Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems},
                            articleno = {427},
                            numpages = {15},
                            keywords = {Visual Metaphor, Design, Creativity Support Tool},
                            location = {Yokohama, Japan},
                            series = {CHI '21}
                            }