ChatGPT vs Bard vs LLaMa Chat (Aug 2023) – AI Chat for CRE Comparison
If you’re an A.CRE reader, you know we’re actively exploring the numerous use cases for generative AI in commercial real estate. As part of those efforts, we’ve used ChatGPT to help us build a custom Excel add-in for CRE professionals, used ChatGPT as an analyst to assess sale comps and create a lease abstract, asked ChatGPT for help in developing Excel logic, leveraged the tool to write a grad school application, explored ways to use AI for interview prep, and more.
The one parallel to each of those use cases thus far, is that we’ve used OpenAI’s GPT-4 (i.e. ChatGPT Plus) as the generative AI engine. Several readers have pointed this out – and the fact that ChatGPT Plus is a paid product – and therefore have asked if Google’s Bard or Meta’s LLaMa language models could be used in lieu of ChatGPT Plus. In response to that question, we thought we’d do a comparison between the three major large language AI models currently available.
I should note that this technology is moving and changing FAST. So, this comparison is as of August 2023 and may (or may not) be relevant in the coming months and years.
Overview of GPT-4 (ChatGPT), Google Bard, and Meta LLaMa
GPT-4, Bard, and LLaMa are multimodal large language models (LLMs) created by OpenAI, Google, and Meta respectively. Each model is trained to understand and generate text in a human-like fashion. However, one significant drawback to large language models is that they occasionally return inaccurate results (known as hallucinations), so users are encouraged to verify any information given to them by tools like ChatGPT, Bard, or LLaMa.
About OpenAI GPT-3.5/4 (ChatGPT)
If you’ve read any of our AI content, this will be a refresher. But GPT-3.5/4, the LLM that powers ChatGPT (3.5) and ChatGPT Plus (3.5 or 4.0), is a state-of-the-art large language model developed by OpenAI. It is the first mover in the space and therefore is the most widely recognized large language model to date. GPT-4 is arguably the most advanced of the large language models currently on the market
The model is trained on a diverse range of internet text, but it doesn’t know specifics about which documents were part of its training set. It’s designed to generate responses to prompts, making it capable of creating anything from stories to code, and it can even answer questions or engage in conversation.
The latest iteration, GPT-4, is a significant improvement over its predecessors, with a larger model size and better training data. It’s more adept at understanding and generating text, making it a powerful tool for a variety of applications, from drafting emails to writing code. It’s also been fine-tuned with reinforcement learning from human feedback, which has helped to improve its performance. Despite these advancements, it’s not without limitations. However, GPT-4 is currently only available to paid users of ChatGPT (called ChatGPT Plus).
About Google Bard
Google Bard is an AI chatbot developed by Google, designed as a competitor to ChatGPT. Like ChatGPT, it allows users to perform a variety of tasks, from planning events and drafting emails to answering complex questions and even coding. Google Bard is likewise powered by a large language model and was initially trained using Language Model for Dialogue Applications (LaMDA) to become conversational. It now also uses Pathways Language Model 2 (PaLM 2) to power more advanced features.
Like ChatGPT, Bard is designed to be conversational. Users interact with it by typing a query or request into a text box, and the AI generates a response in a conversational tone. It can also respond to prompts with images and videos, thanks to its multimodal functionality from PaLM 2.
Google Bard is currently free to use. It’s designed to be a complementary experience to Google Search (and eventually other Google products such as Gmail, Android, etc), and users can click a “Google It” button next to responses when using Bard to access Google Search for more insight. Despite its capabilities, Google Bard (like ChatGPT and LLaMA) has faced criticism for occasionally providing incorrect information and not always citing sources for the information it provides.
About Meta LLaMa 2
First, it’s important to point out that LLaMa 2 is not a Chat interface unlike Bard by Google or ChatGPT by OpenAI are. Instead, it is a large language model created by Meta, the parent company of Facebook, as a competitor to GPT/LaMDA/PaLM 2. However, as of the writing of this post, Meta has not released a chat interface. Nevertheless, I add it to this comparison because various third parties are creating chat bots powered by LLaMa 2 (see Perplexity Lab’s LLaMa Chat).
Unlike the LLMs that power Google Bard or ChatGPT and Bing Chat, LLaMa 2 is an open-source language model. This means it’s freely available for anyone to use for research and commercial purposes. However, Meta has put a condition that any entity with over 700 million users must seek permission before using it.
Despite its capabilities, LLaMa 2 is not available as an end product like ChatGPT. Instead, it’s available for download via Meta’s website for those who know how to work with LLMs. Meaning, if you want to use it you need to use a third-party application to access it (e.g. Perplexity Lab’s LLaMa Chat).
Key Features of ChatGPT, Bard, and LLaMa Chat
ChatGPT – Key Features
ChatGPT is arguably the most feature-rich of any chat interfaces that make large language models user-friendly. However, it reserves many of its features for ChatGPT Plus, the paid version with a cost of $20/month.
The free version provides unlimited access to GPT-3.5 but no access to GPT-4. And while GPT-3.5 is arguably as powerful as either PaLM 2 or LLaMa 2, it falls short when compared to GPT-4, particularly in its ability to reason. This is largely due to GPT-3.5’s shorter-term memory of around 8,000 words, compared to the 64,000-word memory of GPT-4. The free version also includes chat history and some basic settings and allows you to share conversations with others via a unique link.
ChatGPT Plus truly unlocks the power of OpenAI’s large language model. In addition providing the user with access to GPT-4 with its superior capabilities and longer-term memory, it includes ‘Custom Instructions’ which lets you “train” the model on you, the user. This means that the tool will provide responses that speak more directly to you, since it knows something about you.
ChatGPT Plus also includes ChatGPT plugins, which greatly expand ChatGPT’s functionality. Plugins give users access to external databases/services to perform tasks like booking travel, creating a meal plan, playing games, reading and answering questions related to external files such as PDFs or CSV files, and executing code via the Code Interpreter, among others. As of this writing, there were nearly 800 plugins in the ‘Plugin Store’, with hundreds more being created monthly. One drawback to using GPT-4 on ChatGPT Plus is the limit of 50 prompts every three hours.
Google Bard – Key Features
Let’s start with the good – Google Bard is (currently) free to all Google users. It’s also feature-rich, at least compared to LLaMa Chat with chat history, file upload, audio prompting, and an intuitive interface.
Bard offers a unique feature of pinned and recent threads, allowing users to organize their conversations and pick up where they left off. It also enables users to share their Bard conversations with others through shareable links, promoting collaboration and idea sharing. Users can modify Bard’s responses to make them simpler, longer, shorter, more professional, or more casual, providing a tailored experience.
For coders, Bard has expanded its export capabilities for Python code to Replit, in addition to Google Colab. It has also improved its math and data analysis capabilities, detecting computational prompts and running code in the background. Bard-generated tables can now be exported right into Google Sheets, streamlining workflows.
Bard has also introduced location-based responses, providing more relevant information based on the user’s precise location. It can now bring in images from Google Search, enhancing visual communication. Bard has improved its summarization capabilities and made sources more useful by identifying which parts of a response match a source.
LLaMa Chat – Key Features
LLaMa 2 represents a significant upgrade from its predecessor, LLaMa 1. It has been trained with 40% more data, resulting in improved performance across various dimensions including academic ability, reasoning ability, knowledge ability, and comprehension ability. It’s available in three sizes, with 7 billion, 13 billion, and 70 billion parameters, supporting direct dialogue applications and offering flexibility based on the user’s needs.
LLaMa Chat, created by Perplexity, leverages the capabilities of LLaMa 2 and offers a minimalist interface for users. It provides a real-time meter showing the seconds to the first token, tokens per second, the number of tokens used in the response, and how long it took to produce the response. This transparency allows users to understand the model’s performance in real-time. Additionally, it offers the option to copy the response to the clipboard, enhancing the user experience and making it easier to utilize the generated responses.
LLaMa Chat, as of this writing is free.
Comparison for CRE – ChatGPT vs Bard vs LLaMa Chat
So, which AI Chat is right for commercial real estate professionals.
LLaMa Chat vs ChatGPT and Bard
I’ll start with LLaMa Chat by Perplexity. The short of it is that the tool is very much in its infancy. Meta only released LLaMa 2 in July 2023, putting it nearly nine months behind ChatGPT (November 2022) and four months behind Bard (March 2023). On top of that, the chat is created by a third party (Perplexity) who doesn’t seem to have put the same resources into its development as OpenAI or Google have into theirs.
Therefore, LLaMa Chat isn’t really a competitor at this stage. The limited context retention of the underlying language model and minimal features make it a novelty but not a real competitor. I attempted to test the tool by writing some hypothetical clauses for an LOI. After about six responses, for some reason it began giving me advice for how to raise money for a political campaign. Needless to say, we’ll need to give LLaMa Chat more time before taking it seriously.
ChatGPT vs Bard
Now we get to the real competitors: OpenAI’s ChatGPT and Google’s Bard.
Both AI chatbots are designed to generate human-like responses to user prompts, but they differ significantly in their data sources and capabilities. Bard continually draws information from the internet, providing it with the latest information, while ChatGPT’s training data only goes up to 2021, limiting free users to its knowledge of newer research and information. Bard utilizes Google’s newest language model, PaLM 2, while ChatGPT operates on GPT-3.5, with GPT-4 technology available in its paid version, ChatGPT Plus.
Bard excels at answering questions with more relevant information, while ChatGPT shines in generating and summarizing text requests. Bard’s user-friendly interface and real-time information access make it ideal for research, but it can be prone to AI hallucinations (i.e., generating inaccurate information) and lacks the extensive plug-in and integration options of ChatGPT. Conversely, ChatGPT excels at writing, stores previous conversations, and offers a wide array of plug-ins and integrations. However, its responses can sometimes be long and difficult to scan, and access to the more advanced features comes at a cost.
ChatGPT, as the first mover in the generative AI space, and as such boasts a more polished and less bug-ridden interface. It’s evident that Bard, created by a massive corporation, is determined to integrate the tool into Google’s suite of products, complete with a barrage of disclaimers and legal messages. In contrast, ChatGPT, born from a nimble startup, is focused on expanding its connection to external resources via its growing plugin library and is more willing to take risks on new features and functionality.
Having to pay to access the most important features of ChatGPT is a significant drawback compared to the freely accessible Bard. However, in my opinion, the power and capability of those paid features justify the $20/month cost. I personally use ChatGPT Plus daily, and while I give Bard a try every month or so, I inevitably end up back at ChatGPT Plus as it produces a better result.
For a commercial real estate professional, either tool can be effectively used to analyze leases, loan documents, rent rolls, and other real estate material, although inaccuracies do occur – more so with Google Bard than ChatGPT Plus in my experience. Both tools are adept at summarizing emails, drafting emails, or producing offering memorandum or investment committee memo narratives. Bard’s access to real-time information can be incredibly useful, while ChatGPT only offers such functionality with a ChatGPT Plus plugin.
Bard stands out with its human-like responses, user-friendly interface, and real-time internet access, while ChatGPT excels in text-processing tasks like summarization and paragraph writing. However, Bard can sometimes generate inaccurate information which for a CRE professional is a major issue. ChatGPT Plus offers premium features, but it comes at a cost, unlike Bard which is freely accessible.
In short, Google Bard and ChatGPT-4 are both formidable conversational AI tools, each excelling in different areas. And which (or neither) ultimately becomes the dominate LLM we use in the CRE industry going forward is yet to be determined.
If you haven’t already, go ahead and give both a try. As they say, AI won’t (likely) take your job, but the person who learns to use AI tools like these most certainly will.
