The rise of Big Data I want to take you back to 2012 . In those early years, our focus was big data, and everyone else was looking at the cloud. We were pioneering, but in a world that was still beginning to understand the value of data.
Building a Creative Commons dataset for retraining Stable Diffusion using the open source dataset creation framework Fondant
The chatbot landscape is evolving from purely scripted flows to dynamically generated AI-driven conversations. This shift not only enhances the customer experience but also minimizes the maintenance effort required to update these flows while maximizing their robustness and flexibility.
What does the day in the life of a medical specialist who encounters a patient with an unclear diagnosis look like? Its combing through tens or maybe hundreds of scientific papers to find a gene, cell therapy or something else that may be the key to saving their patient’s life. As you can imagine, this can be a lengthy and time-consuming process. But what if there was a tool this specialist could use to get this information through simple queries, cutting down the amount of time it takes to find the needed information?
One of the projects we are working on involves generating code for a custom dialect of a programming language using a large language model (LLM). With a dataset of instructions and their corresponding implementations, we aim to fine-tune a model to automate this process. Given the rapid advancements in AI, fine-tuning LLMs can significantly enhance their performance for specific tasks, offering tailored solutions that generic models might not provide.
Generative AI is ubiquitous these days, and organizations are rapidly integrating GenAI into their business processes. However, building GenAI applications comes with its own set of specific challenges. The models are often large, meaning inference costs for running these models can quickly get out of hand, and model selection often requires balancing performance against costs and latency. Other common challenges include the misuse of generative models or data leakage. While implementing measures such as rate limiting, monitoring, and guardrailing in your GenAI applications can help overcome these problems, doing so for every individual project brings significant overhead for your engineering teams. It also becomes easy to lose track of global usage of generative AI within your organization and leads to many cases of reinventing the wheel as teams solve the same problems over and over again.
In recent years, Large Language Models (LLMs) have revolutionised natural language processing by enabling machines to understand and generate human-like text with unprecedented accuracy and coherence. Their applications span across diverse fields such as chatbots and content creation, driving significant advancements in automation and AI-driven solutions. As a result, LLMs have become crucial tools in both academic research and commercial innovation, pushing the boundaries of what AI can achieve. Though, I’m sure you already knew this.
For avid followers of the Protein design space, you’ll likely have come across our earlier blog detailing the ins and outs of the current state of affairs. Well, time inevitably marches on, and if there are few certainties in this universe, you can count on one being a continual development in Machine Learning. So given the recent advancements, enhancements and brute-forced micro-improvements, ML6 is back once again to give you the scoop on what exactly is “up” in the crazy, little intersection of Protein Design and ML. Though this time there’s a twist… We’ll specifically be focusing on updates in sequence generation, with some other notable mentions where appropriate.
Should we fine-tune a LLM for this use case? Or consider other techniques? At ML6 that may very well be the #1 question we get from customers regarding the use of large language models (LLMs).
Machine learning is a powerful technology with the potential to transform businesses and industries. However, for newcomers, it can be a daunting and time-consuming endeavour. The process of learning the fundamentals, setting up the infrastructure, and understanding the intricacies of ML frameworks can be overwhelming.