HTX intern stories: A Generative AI adventure to relish

 Ignatius Tang shares why his xData internship has been an enjoyable and meaningful learning experience.

HTX intern Ignatius Tang. (Photo: HTX)

Ask Ignatius Tang why he is enjoying his time as an intern at HTX and he’ll likely tell you this is because his work bears similarities to his favourite hobby – gaming.

But this isn’t because his work at HTX is all fun and games. It’s because he gets to – just like in role-playing video games – wander around another universe and make discoveries.

The “universe” that he gets to explore at HTX is the field of artificial intelligence (AI), which the information engineering and media undergrad from Nanyang Technological University (NTU) says he is fascinated with due to the myriad applications this technology possesses.

And it was this interest that compelled him to apply for an internship at HTX’s xData CoE.

“I read online that HTX is working on AI projects like NEMO, which automates the news monitoring process for Home Team officers. I’d initially thought that Generative AI was only useful for drafting emails or creating AI art, but learning about HTX’s work, I was eager to join them and harness the power of AI to help our Home Team officers,” shared the 24-year-old.

Although my strategies weren't always successful, my supervisors always gave me the space to learn from my failures.

The young STEM talent was not let down as his first assignment was to assist in the development of an AI tool that could potentially empower SPF officers. Called Wordsmith, this tool uses Large Language Models (LLMs) to generate and summarise meeting transcripts. Wordsmith also flags out key action items from the transcripts.

The use of Wordsmith would mean officers spend less time on administrative work and more time on pressing tasks.

Deciphering the complex system behind PromptFlow. (Photo: HTX)

While working on Wordsmith, Ignatius also got to learn about another AI tool—a state-of-the-art development platform for LLMs by Microsoft called PromptFlow. With PromptFlow, the xData team can rapidly experiment on LLMs and develop prompts that improve Wordsmith’s speaker distinction capabilities.

However, PromptFlow’s frequent updates by Microsoft meant that Ignatius struggled to find comprehensive guides that could deepen his understanding of the software.

Fortunately, HTX arranged for Ignatius to attend Microsoft's training sessions on deploying PromptFlow.

“I loved the programme as it gave me a better understanding of PromptFlow. This let me perform further tests and experiments to explore how various prompts worked within the Wordsmith model—it was like discovering a new method of exploring LLMs!” he shared.

Evolving expertise

Once Ignatius learnt how to work with LLMs in Wordsmith, he then embarked on a more challenging project involving another AI model called Vision LLMs.

The difference between these two AIs?  While Wordsmith primarily uses transformer-based models for language processing, Vision LLMs contain both transformer-based models and convolutional neural networks, allowing the model to interpret and generate both visual and textual data.

“Working on these Vision LLMs reminded me of why I wanted to join HTX in the first place – this technology could greatly ease the workflow of Home Team officers as they can potentially help analyse multi-page documents in various reports to improve the efficiency and accuracy of administrative and help in verification matters,” he explained.

However, developers of open-source vision models often train their models on data that is not relevant to the Home Team. This meant that Ignatius had the opportunity to shortlist a few open-sourced Vision LLMs and finetune them to suit the Home Team’s needs.

“This was a challenging task because I had to understand how and why the open-source vision models responded differently to certain prompts and our domestic datasets. While it wasn’t easy, I enjoyed experimenting with different prompts and codes to get the exact result I wanted,” he said.

Ignatius hard at work fine-tuning a Vision LLM. (Photo: HTX)

Looking back at his internship journey, Ignatius said he is grateful that he has been given so much room to experiment with various AI solutions.

“My supervisors at xData have never restricted me to a single course of action and this really allowed me to explore various ways of handling difficulties in my project,” he said.

“Although my strategies weren’t always successful, my supervisors always gave me the space to learn from my failures. And that’s one of the biggest takeaways from this internship – failure really is the best teacher!”

Yong Wee_teaser image
Living His Dream
READ MORE
Wen Hui & Caleb
Where innovation impacts lives
READ MORE