AI Word Clouds at COP28 event: A Deep Dive into Our Technology Stack

For the recent COP28 event in Dubai, our client, Microsoft, tasked us with creating an experience (XP) for their booth that featured two screens communicating with each other:

  • The first screen was designed to collect user insights on climate change and provide a branded selfie as a reward.
  • The second screen would visualize the aggregated user insights in a word cloud format.

INSIGHTS XP:

COP Insights

This XP guided users through a series of demographic questions and two questions about their challenges and hopes regarding climate change. Users were then given the option to take a selfie with a branded overlay and download it through a QR code.

To manage the data, we employed Excel to collect the answers, sent a copy to Analytics, and another to the Word Cloud XP via the Web Trigger API.

For the photobooth, we utilized the Webcam asset and the Cloudinary image uploader Interface Asset (IA). This valuable IA, created by @Seb, facilitates image uploads to Cloudinary servers, generating a URL, which we transformed into a QR code for users to download their photos.

WORD CLOUD XP

COP Word Cloud

In this XP, we aggregated and visualized all user input from the Insights XP in a word cloud format. To accomplish this, we employed Excel to store the data and a series of cascading actions to transition from long-form text input to word cloud images. The process involved:

  • Receiving verbose user input via the Word Cloud XP using a Web Trigger IA, which was then stored in Excel.
  • Utilizing a custom Windows version of the OpenAI IA (thanks to @Seb!) with support for GPT-4 to extract the top keywords from the raw input. The prompt looked like this:

Task:
Analyze the provided text and extract keywords.
Include variations of the same word (e.g., singular/plural forms, different tenses) as a single keyword.
Focus on identifying only the keywords that are explicitly present in the text.
Do not invent or add any new keywords that are not in the text.

Output:
Present a list of the extracted keywords, consolidating variations of the same word.
Format the output as a comma-separated list of keywords.
Avoid including any prefixes, titles, or introductions such as “Keywords:”, “List:”, etc.
Example:
If the text contains the words “apple”, “apples”, and “orange”, the output should be exactly: “apple, orange”.

Instructions:
Adhere strictly to the content of the provided text.
Extract and list keywords exactly as they appear, treating variations of the same word as one.
Ensure no additions, extrapolations, or formatting beyond what is specified.

Note to the Model:
This task requires precise and literal execution. Please limit your response to directly reflecting the content of the provided text, considering word variations as one keyword.

  • These keywords were appended to previously extracted keywords and stored in a single Excel cell, representing input from all survey respondents.
  • Since our goal was to visualize a word cloud with only the top 10 keywords per answer, we subscribed to the Word Cloud Generator service found on the RapidAPI marketplace.
  • This API required a block of text as input and returned a PNG word cloud image as output. To ensure the correct weighting of keywords, we extracted the top 10 keywords from the aggregated list and progressively duplicated them in the text block sent to the API. This two step process was done via a combination of AI prompt engineering and Excel formulas.

Forget any previous instruction. Treat this as a unique analysis request. Your task is to analyze the text below, extracting and listing only the top 10 keywords based on their frequency in this specific submission. Each input text includes a unique ID to make it unique and requires fresh analysis. This two-step process was achieved through a combination of prompt engineering and Excel formulas.

Task:
Analyze the provided text for keyword frequency.
Identify and list the top 10 keywords based on frequency.
In cases of frequency ties, determine the order based on the first appearance of the keywords.
Ensure that the analysis is refreshed for each submission, especially when the text length varies.

Output:
Present the top 10 keywords in a comma-separated list.
The order of the keywords in the list implies their rank.
Do not include explicit ranks next to the keywords.
Do not include ID
Limit the list to no more than 10 keywords.
Do not include the unique ID in the output.

Expected Output Format:
keyword1, keyword2, …, keyword10

Note: If fewer than 10 keywords are present in the text, list only those available. The order of the keywords should reflect their frequency and appearance in the text.

  • The final step involved visualizing the word cloud. We utilized an HTML Frame Asset to combine the API input and display the word cloud PNG output. We then overlaid various graphics on it to seamlessly integrate the word cloud with the rest of the branded UI.

CONCLUSIONS:
In conclusion, the ability of Intuiface to integrate diverse technologies is truly impressive. Although I’m not a developer, some aspects of this project required a deeper understanding of HTML and JavaScript, which I didn’t possess. Fortunately, with the assistance of ChatGPT, we were able to overcome several technical challenges.

I also highly encourage anyone to experiment with the OpenAI IA made available by Intuiface. With proper prompt engineering (a no-code skill), you can empower your XP with exceptional capabilities.

3 Likes

@tosolini Fantastic execution. I am also spellbound by those prompts. :slight_smile: How much of those prompts are the result of trial and error, trial and error, trial and error… until it comes out right, and how much (if any) are you using the assistance of an AI prompt generation service?

Thanks @Geoff. The prompting part took me several hours to get it right. It started with trial and error, until I got it 95% right and I ran out of ideas to fully finalize it. Then I asked ChatGPT to help me fine tune the wording, followed by more trial and error.

Just to name some aspects of this project that made things more complex.

  • I needed an OpenAI Interface Asset that supported GPT-4 in Player for Windows (thanks @Seb for building it)
  • Excel has a limit of 32K characters, so I was worried to reach it by aggregating cumulative keywords in one cell
  • Luckily OpenAI announced recently a model that would process bigger amount of text, so I used gpt-4-1106-preview
  • I created a hidden Playground UI so I could cut testing time (that’s what you see in the animated gif above)
  • GPT-4 didn’t process correctly new keyword entries, until I added some sort of unique ID in front to make each keyword set different enough from the others. That I learned asking ChatGPT
  • I realized that some actions weren’t reliably executed by AI (e.g. duplicating keywords to create their weight before passing them to the Word Cloud API). So I had to remove that functionality and recreate it in Excel

Anyway, there is so much potential in combining these tools together.

1 Like

We’ve definitely taken to thinking about how LLM integration could take shape within Intuiface. Heck, automatic Interface Asset creation? The problem is this notion of reliability. And worse, it’s hard to know where it’s unreliable and/or will hallucinate. Even if it’s right 95% of the time, that 5% is hard to tolerate.

Have you reached any general conclusions about the types of issues GPT struggles with?

1 Like

As you mentioned, hallucinations and unreliable repeatable results might be a challenge, especially if you let AI choose for you e.g. the filter of a database, or the description of a topic.

There are ways to mitigate the risk, by forcing AI to stay within its domain of knowledge. The risk is lower when it comes to generating images like abstract backgrounds.