How to Learn AI on Your Own (a self-study guide)
The Myth Of Large Language Models
Presented by Zscaler
In 2023, ChatGPT started a technological revolution. Simple interactive AI agents quickly moved into indexing documents, connecting to data sources and even data analysis with a simple sentence. With that success, a lot of people promised last year to deliver large language models (LLMs) soon … and very few of those promises have been fulfilled because:
In this article, we'll examine the role of AI agents in linking LLMs with backend systems. Then, we'll look at what's setting up AI agents as the next generation of user interface and user experience (UI/UX). Finally, we'll talk about why we need to bring back some principles of software engineering that people seem to have forgotten lately.
I want a pizza in 20 minutesLLMs offer more intuitive, streamlined UI/UX than traditional point-and-click interactions. Let's suppose you want to order a "gourmet margherita pizza delivered in 20 minutes" through a delivery app.
Using normal UI/UX, this seemingly simple request could require multiple complex interactions and take several minutes. You'll probably have to:
LLMs like GPT-3 have demonstrated exceptional abilities in natural language processing (NLP) and generating coherent, relevant responses. Connecting external data sources, algorithms and specialized interfaces to an LLM gives it even more flexibility and analysis capabilities, even opening the door to tasks not yet possible with today's LLMs.
LLM-based interfaces can be highly complex. Even our "pizza" request needs to connect multiple systems — restaurant databases, inventory management, delivery tracking and more — to complete the order. Yet more connections will be needed to enable greater flexibility, which is crucial to ensure a seamless experience for diverse requests. LLMs can't do all that by themselves.
AI AgentsLLMs serve as the foundation for AI agents. To respond to a diverse range of queries, an AI agent leverages an LLM in conjunction with several integral auxiliary components:
This white paper provides a more complete description of AI agents and components.
Adding LLM-based AI agents to your data is an engineering problem, not a research problemNatural language can make it much easier to specify use cases for software development. However, the ambiguity of the English language is leading to a new problem in software development, where systems aren't well specified or understood.
Fred Brooks outlined many central software engineering principles in his 1975 book The Mythical Man-Month, some of which people seem to have forgotten during the LLM rush. For instance:
No silver bullet. No singular development will eliminate the need for proper software engineering practices. Not even LLMs.
The manual and formal documents. Thanks to hype, this is probably the most forgotten principle in the age of LLMs. It's not enough to say "develop a system that will tell me how to order things like a gourmet margherita pizza in 20 minutes." This requires documentation of a whole array of other use cases, required backend systems, new types of visualizations to be created, and — crucially — specifications of what the system will not do. "Things like" seems to have become a norm in LLM software development, as if an LLM can magically connect to backend systems and visualize data it has never learned to understand.
(We look at several more of Brooks' points over on the Zscaler blog.)
In a recent white paper, we've addressed the issue of lack of proper specification of software systems, and showed a way we can create formal specifications for LLM-based intelligent systems so they can follow sound software engineering principles.
Bad dataEnsuring LLM-based AI agents work effectively requires more formal data organization and writing methodologies, as discussed on LinkedIn. On top of that, LLM-based systems expect well-written documentation. OpenAI recently stated that it is "impossible" to train AI without using copyrighted works. Essentially, training these models requires not only a tremendous amount of text, but also high-quality text, such as many copyrighted, published works.
Well-written text is even more important if you use RAG-based technologies. In RAG, we index document chunks using embedding technologies in vector databases, and whenever a user asks a question, we return the top-ranking documents to a generator LLM that composes the answer.
ConclusionsThere's an explosion of LLM-based promises in the field, and very few are coming to fruition. In order to realize those promises and build intelligent AI systems, it's time to recognize we are building complex software engineering systems, not prototypes.
LLM-based intelligent systems bring another level of complexity to system design. We need to consider what it will take to specify and test such systems properly. Finally, we need to treat data as a first-class citizen, as these intelligent systems are much more susceptible to bad data than other systems.
You can read an unabridged version of this article on the Zscaler blog.
Claudionor N. Coelho Jr. Is Chief AI Officer at Zscaler.
Sree Koratala is VP, Product Management, Platform Initiatives at Zscaler.
Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they're always clearly marked. For more information, contact sales@venturebeat.Com.Simple Explanation Of Model Basin Methods
The primary work of an experimental model basin is the estimation, with reasonable accuracy, of the resistance and horse-power of full-sized ships from experiments with small and inexpensive models. The first experimental model basin was built nearly forty years ago by Mr. William Froude in his garden near Torquay, England. It was soon taken over by the Admiralty. The second was built by an enterprising firm of Clyde shipbuilders at Dumbarton, Scotland, a little more than twenty years ago. During the last ten years the number of model basins has increased rapidly. Besides England, there are now government basins in Italy, Russia, the United States, Germany, and France, and there are private basins iIi England, Germany, and the United States. In the United States the government basin at Washington began work in 1899. There is a private basin at the University of Michigan, and at Cornell University a canal used for hydraulic experiments is also used, to some extent, for experiments with models of ships and propellers. Existing basins are from 300 to 550 feet long, from 20 to 45 feet wide, and from 8 to 14 feet deep. All of these experimental basins rely upon the principles first enunciated and applied in practice to vessels by Mr. William Froude. The law of Comparison, or Froude's law, as it is frequently called, teaches us that certain resistances of ships and models 11t corresponding speeds vary directly as their displacements. That is to say, at speeds proportional to the square roots of any similar linear dimensions (preferably length), resistances which obey Froude's law are directly as the displacements or as the cubes of similar linear dimensions. Mr. Froude demonstrated conclusively the value of his methods by showing that the actual resistance of a full-sized ship— the English naval vessel "Greyhound"— as determined by towing her, agreed very closely with resistance estimated by his methods from the resistance of a smll model. There are three main components of the resistance of a ship, viz., the skin friction, or the resistance due to the rubbing of the water on the surface; the eddy resistance, or resistance due to the formation of eddies, such as those behind stern-posts, struts, etc.; and the wave- making resistance, or the resistance due to the creation of waves as the ship advances through the water. The first and the third of these elements are the more important in practice. The eddy resistance in properly designed ships is not great, and for prac- ' tical purposes is classed with the wave- making resistance, the two combined forming what is often called the residuary resistance. It is the residuary resistance to which the law of comparison is applied directly. The skin friction does not follow the law of comparison, but, fortunately, it can be estimated with reasonable approach to accuracy for both model and ship. The quantities that have to be dealt with in model experiments are small, and it is necessary that the models be constructed with accuracy and tested by a reliable apparatus with much care. The majority of model basins use paraffin wax in the construction of models, following the practice originally established by Mr. Froude. Paraffin cannot be made to stand the summer heat of Washington without inadmissible change of form, so that at the United States basin wood is used as material for the models. This is more expensive, but, otherwise, has everything in its favor. It enables models 20 feet in length to be regularly used, while with paraffin from 12 to 14 feet is the greatest practicable length. Whether of paraffin or wood, a model must represent accurately to scale the under-water body of its corresponding ship and have a thoroughly smooth surface. At the Washington basin the principal shaping of the models is done by machinery, finishing touches being put on by hand. When preparing for testing, the model is carefully weighted in the water to the exact displacement and trim corresponding to the full-sized ship, and then towed at various speeds, through recording dynamo- metric apparatus, from a carriage which runs back Fig. 1.-Curves of Resistance and Change of Level of Model. Length, 20 Feet; Breadth, 2.076 Feet; Draft, 0.863 Feet; Displacement, 1,694 Pounds. Fig. T.-Speed of Ship About 18.5 Knots. Cumparison of Wave Profile Fore and Aft Estimated from Model Runs with That Observed on Trials of Shi". Fig. 3.—Curves of Effective Horse-Power for Ship of 40,000 Tons Displacement, 800 Feet Long and 0 5.5, 0.60, and 0.6.5 Cylindrical Coefficient. SIMPLE EXPLANATION OF MODEL BASIN METHODS. And forth, and is arranged so that it can tow models at a large number of speeds, the speed of each run being accurately determined and recorded. At the Washington basin electricity was used for the first time to drive the towing carriage. This feature has been copied in all basins of later date. The recorded resistances are plotted as ordinates above the speeds, thus giving a number of spots through which a fair average curve is drawn, giving the total resistance of the model. Fig. 1 shows from an actual model test a number of spots and the resistance curve RR drawn through them. The model represented a collier with a long parallel middle body and was tested to a corresponding speed higher than such a vessel could obtain in service. Its resistance curve shows up well the "humps" and "hollows" which are due to the fact that at some speeds the waves caused by the bow accentuate those caused by the stern, giving a "hump," while at other speeds the waves from the bow partially neutralize those at the stern, causing a "hollow." The curves in Fig. 1 showing the change of level of bow and stern are typical. In practically every case, vessels of normal types settle bodily both forward and aft, as speed increases, with but little change of trim until, at a critical speed, which is in knots about 1.15 the square root of the length in feet (5.25 knots for a 20- foot model), the bow begins to rise sharply and the stern to settle sharply. Not many actual ships are fast enough to reach this critical speed. The displacement, wetted surface, and all other necessary quantities in connection with the model having been calculated in advance, the first step in the reduction of the results is to plot a curve such as FF in Fig. 1, which, being the frictional resistance of a plane 20 feet long and having the same surface as the model, is taken as representing the fric- tional resistance of the model. It is important that the surface be taken as 20 feet long, because it has been found by numerous experiments that the coefficient of friction of a smooth plane surface in water varies with the length; the greater the length the smaller the coefficient of friction. The index or power of the speed according to which the friction varies also changes somewhat with the length; but for lengths such as those of actual ships the index is practically constant at 1.83. The intercepts in Fig. 1 between the curves RR and FF give the residuary resistance. The application of the law of comparison to this residuary resistance is comparatively simple. Thus, at 4 knots model speed the residuary resistance is 7.73 pounds. Suppose the 20-foot model represents a ship 500 feet long. Then the linear ratio between model and ship is 25. The displacements are as the cubc of the linear ratio, or as 15,625. Corresponding speeds will be as V 25, or ship speed will be 5 times model speed. The residuary resistance of the ship in fresh water would be, then, 15,625 x 7.73, or, in round numbers, 120,800 pounds at 5x4 = 20 knots. As a general thing, however, while model results are obtained in fresh water, we wish to know the resistance of a ship in salt water. All resistances are taken to vary as the density of the water, which is an ample approximation. So, to obtain the resistance of the ship in salt water, we multiply 120,800 pounds by the ratio between fresh and salt water, which is 1.026. Thus we determine the residuary resistance of the ship in spit water to be 123,900 pounds. We need still to calculate its frictional resistance. We know, by calculation, its wetted surface, and for the coefficient of friction we use Froude's or Tideman's frictional coefficients, obtained from experiments with plane surfaces many years ago, and which, ii may be remarked, have been confirmed by such experiments as have been made on the subject at the United States model basin. But some experiments on large and long planes, with snrfaces such as commonly presented by ships' bottoms, towed at high speeds, would be of great value. The coefficients in use are deduced from experiments with smooth planes of comparatively small dimensions and towed at comparatively low speeds. Actual ships' bottoms are seldom as smooth as pir.Nes are made, and their dimensions are much greater- than the planes hitherto tested. The needed investigation. However, would have to be car- continued on page 420.)
This USC Professor From Kolkata Is On A Journey To Civilise AI
The present-day AI system is smitten with hallucinations – for some, it is a feature (and not a bug), and for others, it's frustration. For Amitava Das, a professor at the University of Southern Carolina, it was a worry that lead to the creation of what he calls 'Civilising AI or Constitutional AI'.
"AIs [LLMs] are very smart and at the same time, they are also stupid. And we do not know how to handle these in either case. On the other hand, AI-generated content becomes so eloquent that it is impossible to detect it. No Turing test is able to do it," said Das, in an interview with AIM, sharing the origin of Civilised AI, a framework to proactively measure the hallucination of a model, alongside mitigating and managing risks associated with it.
With a PhD in natural language processing (NLP) from Jadavpur University in Kolkata, Das has been working in the field for more than 20 years, which he says was all because of his passion for working in AI ever since the beginning. Having two academic postdocs from Europe and the USA, he is currently a professor at the Artificial Intelligence Institute of South Carolina (AIISC).
Das started the AI research lab at Wipro and is still an advisor there. Having worked across geographies, and along with IITs and universities world over, he has also created a joint PhD program for industry practitioners to earn doctorate degrees. He's currently focused on reducing hallucinations within language models.
There are several research papers and models coming up to solve the AI detection problem. GPTZero, DetectGPT, and RoBERTa baseline are all trying to do the same, but a lot still needs to be done. The most recent paper, Ghostbuster, also works on solving the same problem.
That is why Das and his team coined the term 'Civilised AI' to explain how to handle the problem of hallucinations, and at the same time make it easier to detect if a text is AI generated, and tackle the misinformation and the possible harm it poses to the society.
Single metric to civilise AIUsing the example of Google's Bard and how the company lost around $100 million because of its hallucinations, Das emphasised the need to control what AI is writing. "On the one hand, we need AI to be creative when it comes to fields like journalism as it helps in writing. But on the other hand, if I am a lawyer working on a legal case, hallucination is not allowed at all," he explained.
Das's paper, titled The Troubling Emergence of Hallucination in Large Language Models – An Extensive Definition, Quantification, and Prescriptive Remediations, addresses this problem of balancing hallucinations with eloquence within AI models, along with 'Counter Turing Test CT^2: AI-Generated Text Detection is Not as Easy as You May Think — Introducing AI Detectability Index', the paper that won the Outstanding Paper award at EMNLP 2023.
"We call this dichotomy a factual mirage and silver lining, how much of each is allowed," explained Das, and that is what led his research team to introduce new metrics called AI detectability index, AI adversarial attack vulnerability Index, and AI hallucination index.
"AI is just a balance between these three," he added that there is still not a clear-cut solution for this, but the team is trying to derive a formula for this, which is called the Civilising Index, an amalgamation of all of this.
He said it is a difficult scenario to tackle, illustrating with Einstein's mass-energy equivalence relation, which led to the creation of the atomic bomb. He also gave examples of other AI-related research that needs to be regulated if it involves risk to human lives such as bioinformatics.
"It is the people who created the atomic bomb and the people who created atomic energy. So there are people who should make AI models better, and then there is my team who will make better detection techniques," he added.
"Regulation has to come to specific use case, not in a broad way," he asserted. Each field such as medicine, military, legal, etc need to have separate licences on how they can use AI to tackle the hallucinations, leading to emerging risks within the systems.
The need for Indic LLMsDas is also passionate about how language models can be made for Indic languages and has released a code-mixing model called CONFLATOR for the purpose. He started working in this field around eight years back and is regarded as one of the pioneers of the field by Microsoft Research.
He explained that the problem that comes with Indic languages and datasets is that it is very hard to understand. "Hindi is getting mixed into English or English is getting mixed into Hindi," Das explained about how it is very difficult to understand the switching point in languages as most of the data is from the Internet.
"That is why we introduced Conflator," he added, which is still a complicated field that deals with introducing electromagnetic waves into a neural network for code-mixing for positional encoding formulation.
When it comes to Indic language models "we do not have a lot of monolingual data in India. This discussion has been going on between NITI Aayog and IIIE for several years, before the launch of ChatGPT and others," Das was part of the discussions with the Indian government for making Indic language models possible.
That is why he believes that Indian models are still far behind English models in terms of power and detectability. Das told AIM that he is closely working with the Indian government and IIIT Hyderabad and is soon going to publish a paper on the topic.
Comments
Post a Comment