Hub & Spoken.

What's in this podcast?

In this episode, Jason talks to Bill Inmon, a veteran expert in the field of using language in data and data warehousing. Together they discuss the complexities of drawing out meaning from text and the importance of preserving its emotion and context.

Listen to this episode on Spotify, iTunes, and Stitcher. You can also catch up on the previous episodes of the Hub & Spoken podcast when you subscribe.

What are your thoughts on this topic? We’d love to hear from you; join the #HubandSpoken discussion and let us know on Twitter and LinkedIn.

For more on data, take a look at the webinars and events that we have lined up for you.

One big message

As a human race that communicates with each other everyday, we often take for granted how unique and complex language can actually be. Overall, understanding the rules of language in data is crucial for effectively processing and analysing it. However, it can be a complex task as language is constantly evolving and has many nuances.

01:23 Bill’s background in data and data warehousing

02:30 The complexities of using text in data

07:50 How language can be improved to help support things such as research

12:13 The challenges of using text in data

16:04 What kind of structure is needed to help sort language into data

21:54 The progress of leveraging data from text

26:20 Use cases where data has been successfully pulled from mass amounts of documents and text

30:16 The impact of social media using Nike as a case study

38:10 The future of text in data

The rules of language

There are many complexities in language, with each different language offering its own set of rules. As a human race, we often take for granted all the nuances that come with language that AI and technology is trying to catch up with. 

When working with text data, it is important to understand the rules of language need to be specifically dictated in order to effectively process and analyse it. This includes understanding grammar, punctuation, sentence structure, and vocabulary. Additionally, it is important to be aware of cultural context and idiomatic expressions, as these can greatly impact the meaning of the text.

Machine learning and text

Machine learning models can also be used to analyse language data, with many people trying to find the ‘magic algorithm.’ But it is important to understand the limitations of these models. For example, they may have difficulty understanding sarcasm or idiomatic expressions.

The rise of Artifical Intelligence based language models such as ChatGPT, Siri, Alexa and other advanced language models, has greatly improved the ability of machines to understand and process natural language. However, even with these advancements, it is important to remember that there are still limitations and challenges when it comes to language understanding. It is essential to continuously evaluate and improve the models to increase their accuracy and ability to understand the nuances of language. Additionally, it is important to consider the ethical implications of using these models, such as potential biases and the possibility of misuse.

Maintaining the fluidity of language

In order to understand anything you need context. When you look at numbers, usually the context is quite straightforward. But when you read text, 90% of the intellectual battle is with context, not text. 

There is no magic algorithm that can help provide context to text. This is because there are limitations in the capability of machines to understand the nuances and subtleties of human language. The best way to understand context is through human interpretation but this is extremely time-consuming. Instead, data scientists rely on a network of different algorithms to try and help analyse and distil large amounts of language data while trying to keep the meaning of the text intact. 

Trying to leverage data from text for commercial use

It is one thing to study text and understand the intricacies of language, but it is another thing entirely to be able to effectively utilise this information for commercial or research purposes. In order to do so, it is important to have a deep understanding of the context in which the text is written, including cultural and social factors, as well as the intent and tone of the text.

Large corporations often have vast amounts of language data in the form of documents, and one of their main challenges is being able to quickly and easily find the information they need. They often use cataloguing and organisation methods based on general topics, but this may not fully capture the nuances of the text. 

On the other hand, social media presents a unique set of challenges and opportunities when it comes to using language data. Users on social media frequently use sarcasm, jokes, and bend the rules of language, making it difficult for machines to fully understand the meaning of the text. However, social media has also broken down barriers and has given people the ability to share their thoughts and opinions on products and services in real time, which can provide valuable insights for businesses. 

Conclusion

Language is an extremely tricky thing to distil down into simple data.  However, having an appreciation for its complexities can help us better leverage data and create models that will enable people to quickly gather large chunks of text and transform it into something that is more accessible.