Introduction to Testing AI Models, LLMs and Chatbots

Introduction to Testing AI Models, LLMs and Chatbots, available at $54.99, has an average rating of 4.82, with 72 lectures, based on 11 reviews, and has 38 subscribers.

You will learn about Understand how AI is working Understand basic software testing Understand how AI is tested compared to traditional software Gain knowledge on testing for ethics Understand how to test reasoning abilities of AI Understand functional AI testing Gain know how on how to validate NPL See how to benchmark AI against models HellaSWAG, MMLU, CODEXGLUE, BLEU, Humaneval Importance of test data and model drifting See how chatbots can be tested with real chatgpt examples This course is ideal for individuals who are Citizen Developer or Software testers or quality engineers It is particularly useful for Citizen Developer or Software testers or quality engineers.

Enroll now: Introduction to Testing AI Models, LLMs and Chatbots

Summary

Title: Introduction to Testing AI Models, LLMs and Chatbots

Price: $54.99

Average Rating: 4.82

Number of Lectures: 72

Number of Published Lectures: 68

Number of Curriculum Items: 72

Number of Published Curriculum Objects: 68

Original Price: $49.99

Quality Status: approved

Status: Live

What You Will Learn

Understand how AI is working

Understand basic software testing

Understand how AI is tested compared to traditional software

Gain knowledge on testing for ethics

Understand how to test reasoning abilities of AI

Understand functional AI testing

Gain know how on how to validate NPL

See how to benchmark AI against models HellaSWAG, MMLU, CODEXGLUE, BLEU, Humaneval

Importance of test data and model drifting

See how chatbots can be tested with real chatgpt examples

Who Should Attend

Citizen Developer

Software testers

quality engineers

Target Audiences

Citizen Developer

Software testers

quality engineers

Welcome to “Testing AI: Foundation Models, LLMs, Chatbots & More,” your comprehensive guide to understanding the fundamentals of testing advanced AI systems. Whether you’re a developer, a data scientist, or simply an AI enthusiast, this course will equip you with the knowledge and skills necessary to assess and improve the reliability, performance, and safety of AI technologies.

What You Will Learn:

Introduction to AI Testing: Understand the importance of testing AI systems, including ethical considerations and the potential impacts of AI failures.

Testing Basics: Learn about different types of testing methodologies like unit testing, integration testing, and system testing as applied to AI.

Special Focus on Foundation Models and LLMs: Dive deep into the challenges and techniques for testing large language models and foundational AI systems that are reshaping numerous industries.

Chatbot Testing: Explore the unique aspects of testing conversational AI, ensuring they respond accurately and appropriately in varied scenarios.

AI System Evaluations: Learn how to design and implement effective testing regimes for different AI-based systems, using both manual and automated tools.

Performance Metrics: Understand the key performance indicators (KPIs) and metrics essential for evaluating AI system efficiency and effectiveness.

Case Studies: Gain insights from real-world scenarios that highlight common pitfalls and best practices in AI testing.

Ethical AI: understand the risk with AI and the ethics behind AI. How can and should you test for this

Benchmarking: Understand how to test the AI against some common benchmarking models such as: BLUE, HellaSWAG, MMLU, CODEXGLUE, HumanEval

Who This Course Is For:

This course is ideal for anyone looking to gain a solid grounding in the techniques and practices essential for testing AI systems. Whether you’re starting a career in AI, looking to enhance your professional skills, or interested in the mechanisms behind AI system reliability, this course has valuable insights for you.

Course Features:

Engaging video lectures

Practical assignments and hands-on projects

Quizzes and exams to test your knowledge

Access to a community forum for discussion and collaboration

Lifetime access to course materials

Enroll now to start mastering the crucial skill of testing AI systems and ensure you’re prepared to contribute to the development of safe and reliable AI technologies!

Course Curriculum

Chapter 1: Introduction

Lecture 1: Introduction to Material

Lecture 2: Introduction to Generative Artificial Intelligence

Lecture 3: Demo On AI Capabilities

Lecture 4: History of AI from 1950 to 2024

Chapter 2: Setting up Environment

Lecture 1: Install VS Code

Lecture 2: Installing python

Lecture 3: Install Python Dependencies – PIP

Lecture 4: Install NodeJS and NPM

Lecture 5: Get an OPENAI API Key

Chapter 3: 7 Used LLM Testing Benchmarks

Lecture 1: Introduction to Benchmarking for LLMs

Lecture 2: 7 Benchmarking Models

Lecture 3: TruthfulQA – Truthfulness

Lecture 4: Python – Demo – Benchmarking Open AI CHAT GPT – TruthfulQA

Lecture 5: Python – Demo – Benchmarking Open AI CHAT GPT – MMLU

Lecture 6: Python – Demo – Benchmarking Open AI CHAT GPT – HumanEval

Chapter 4: Introduction to Artificial Intelligence

Lecture 1: What makes up AI

Lecture 2: Where do Large Language Models(LLM) fit into AI

Lecture 3: Natural Language Processing

Lecture 4: Types of Machine Learning

Lecture 5: Importance of Training Data

Lecture 6: Machine Learning – Supervised ML

Lecture 7: Machine Learning – Unsupervised ML

Lecture 8: Machine Learning – Reinforced ML

Lecture 9: Neural Networks and Deep Learning

Lecture 10: Importance of Training Data

Lecture 11: What is a Large Language Model – LLM

Lecture 12: Generative AI – What it is

Chapter 5: LLM Functional Testing – Traditional Software Perspective

Lecture 1: Types of Testing in Software

Lecture 2: Testing Types for LLMs | Foundation Models

Lecture 3: Overall Testing Approach to LLMs

Lecture 4: Basic Content Generation for LLMs

Lecture 5: Temperature Testing of LLMs

Lecture 6: Functional Completeness

Lecture 7: Functional Correctness

Lecture 8: Accuracy Testing

Lecture 9: Repeatability Testing

Lecture 10: Multimodal Testing

Lecture 11: Efficiency and Simplicity Validations

Lecture 12: Learning Ability- Chat GPT and Google Vertex AI

Lecture 13: Statistical Correctness

Lecture 14: LLM AI Testing – Long-Term Drift Testing

Lecture 15: [Demo ] – Vertex AI Learning Ability – Model Training – Manual

Lecture 16: [Demo ] – Vertex AI Learning Ability – Model Training – Json

Chapter 6: LLM AI Reasoning | Intelligence Testing

Lecture 1: Creative Logical Abilities – A bit of fun

Lecture 2: Reasoning – Causal Reasoning

Lecture 3: Reasoning Ability | Deductive | Abductive | Inductive Logic

Lecture 4: Reverse Reasoning

Lecture 5: Counterfactual Reasoning

Lecture 6: Abstract Reasoning

Lecture 7: Generative Reasoning

Chapter 7: NLP – Style and Emotions

Lecture 1: Text Capability – Named Entity Recognition (NER)

Lecture 2: Text Capability – Style Transfer

Lecture 3: Text Capability – Sarcasm and Humor Detection

Lecture 4: Text Capability – Discourse Coherence

Chapter 8: Chatbot Testing

Lecture 1: Understand Task Based vs AI Based Chatbots

Lecture 2: Understanding Chatbots based on LLM

Lecture 3: Functional Testing for chatbots

Lecture 4: Context and Memory Testing

Lecture 5: Blabber Testing | Context forcing

Chapter 9: Testing of Ethical AI – LLM Non functional Testing

Lecture 1: EU Regulation of AI Systems

Lecture 2: AI and Biases

Lecture 3: GEN AI and Privacy

Lecture 4: GEN AI and Intellectual Property

Lecture 5: Hallucinations

Lecture 6: Gen AI and Misinformation or Disinformation

Lecture 7: Google Moderation Service

Lecture 8: OPENAI-CHAT GPT Moderation Service

Lecture 9: GEN AI and Deep Fake

Instructors

Dan Andrei Bucureanu
Quality Transformation Consultant

Rating Distribution

1 stars: 0 votes

2 stars: 0 votes

3 stars: 1 votes

4 stars: 1 votes

5 stars: 9 votes

Frequently Asked Questions

How long do I have access to the course materials?

You can view and review the lecture materials indefinitely, like an on-demand channel.

Can I take my courses with me wherever I go?

Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don’t have an internet connection, some instructors also let their students download course lectures. That’s up to the instructor though, so make sure you get on their good side!