Storyboarding with Midjourney

With a background in textile design, I have been using Midjourney since two years ago. Midjourney is an AI-driven tool that transforms text prompts into images based on your discription words. Midjourney has been an inspiration for me, it helped me developing my thoughts from a different perspective, what it generates can be exactly what I want or vice versa. It can be very dependent on one’s output.

Recently, we had a group project for creating a filming project, I had the role of creating the story board. With time consuming, and I am honestly not a very good drawer, I thought why not try with Midjourney to create a more realistic story board. It was challenging, A simple sentence can spiral into complexity when you’re trying to get the perfect result, and finding the right descriptive words is crucial. The outcome was not always how I imagined, but it worked well for everyone in the team to understand.

At first, creating a continuous storyline felt impossible. Midjourney is designed to generate individual images from separate prompts, not a cohesive sequence of visuals. But as I got the hang of it, things became easier. I learned to keep the style consistent (black and white in this case) and use reference images to maintain continuity for characters and settings. Patience, practice, and trial-and-error were key. Interestingly, even the “wrong” outputs sometimes sparked creative ideas that I hadn’t thought of before.

Figure 1: “at the meeting point outside the restaurant—4 college girls say hi and hugging—black and white.”
Figure 2: “finally four girls met and sit, chatting, laughing, gossiping in the cafe—cref https://s.mj.run/wKRfEhsWZiQ.”

Few issues drew my attention during generating the story board. Midjourney defaulted to young, white characters without me mentioning anything about race and age, which reflects a deeper issue of biased training data. Seems like while AI sourcing data from the internet, likely mirrors gender norms and hierarchies where certain groups are more represented in the visual culture. Without giving specific information and instructions, AI defaults to mainstream culture and representation in the societal system. These datasets often lack detailed knowledge which made sense when I give specific knitwear terminology while doing design works in the past, it seems generating designs based on more popularized knitting ideas. AI has significant limitations and leading gaps in understanding, as it not always understands or generates results aligned with my ideas, which aligns with Ruha Benjamin’s analysis on how AI encode biases.

Another challenge was maintaining continuity between Figures 1 and 2. For instance, when I try to generate an image base on the character from figure 1, the characters in figure 2 seems wearing the same outfit, their clothes are oddly inconsistent. In addition, the style of the image shifts to a hand-sketched look without telling so, which points out another limitation of AI – difficult in retain context or continuity across generations. From my understanding, this happens because AI models aren’t inherently designed for sequential tasks like storyboarding. They interpret each prompt in isolation, which means users have to fill the gaps of continued prompts by experimenting with prompts and reference images.

Despite the challenges, the creative “wrong” outputs highlight the fact that AI can be a partner in creativity providing inspirations rather than just a tool to generate one’s thoughts. Mistakes or misunderstandings can lead to surprising designs or ideas unintentionally, thinking it as institution expands one’s creative capacity. It reflecting the potential of serendipitous creativity through collaboration with AI.

For me, this journey has been both frustrating and fascinating. It’s remarkable to see how much AI can help with creativity, but it’s clear there’s limitations on inclusivity, intuitive, and the ability to align with diverse perspectives. For now it is also something tells that AI can’t replace human creativity, but I see potentials in AI tools.