Python is simpler compared to many programming languages, however, there are still challenges (like syntax, and writing code correctly, and debugging) that students, or non-technical people, can take time and persistence in learning.
AI tools can be beneficial as a practice companion for learning process. AI can help explain a concept in plain English, provide an example, and even provide code you can practice with quickly. For example, the tool QuickGen AI can demonstrate the use of a function, explain how loops work line by line, and suggest exercises for students to practice the concepts, which is all much quicker so students learn faster.
When people are stuck on small coding issues and problems, you can rely on AI for assistance, because AI can review code for errors, provide fix, and then explain why the code still did not work. In addition, AI tools can create small coding projects, like calculators, games, or simple chat bots, to practice programming concepts in Python. Thus, AI provides instant feedback much quicker than needing to research on your own or work through online tutorials.
Python is a programming language that is clear, easy to learn, and very popular among both beginners and professionals. It has clear syntax which means the code is easy to read and write. Python is used in web development, data analysis, automation, and of course artificial intelligence.
AI is all about working with data and learn patterns from that data and then make decisions based on patterns. Python is perfect for this because it has tons of excellent libraries for working with AI, such as: TensorFlow, PyTorch, Scikit-learn and Keras. These libraries allow you to build AI models, without writing everything from scratch, which saves time and effort.
Using Python you can collect data, clean it, analyze it and then create AI models that learn from the data. You could build chatbots, recommendation engines, image recognition tools or even AI writing assistants. Python is likely the best choice if you are building AI models, because it makes all these tasks easier and faster for you as a beginner or as a professional.
Since manually reviewing these papers would be (very) time-consuming, let’s see how AI can help. One could build a tool that analyzes the contents of each PDF on my desktop and organize them into folders based on topics. Text embeddings can translate each paper into a dense vector representation, from which similar articles could be clustered using a traditional machine learning algorithm like K-Means.
Read the abstract of each research article using PyMuPDF Use the sentence-transformers library to translate abstracts into text embeddings and store them in a Pandas DataFrame Use your favorite clustering algorithm from sklearn to group the embeddings based on similarity Create folders for each cluster and move the files into the appropriate folder.
Libraries: PyMuPDF, sentence_transformers, pandas, sklearn
from sentence_transformers import SentenceTransformer
# load embedding model
model = SentenceTransformer("all-MiniLM-L6-v2")
# store abstracts in a list
abstract_list = ["abstract 1", "abstract 2"]
# calculate embeddings
embeddings = model.encode(abstract_list)
A couple of months ago, I helped a company create a basic RAG system for a set of technical reports. One of the challenges with searching such reports is that key information is often presented in plots and figures rather than text.
Given a PDF, chunk it into sections and extract the images using PyMuPDF Use a multimodal embedding model (e.g. nomic-ai/nomic-embed-text-v1.5) to represent the chunks and images as dense vectors and store them in a dataframe Repeat for all PDFs in the knowledge base Given a user query, pass it through the same embedding model used for the knowledge base Compute the cosine similarity score between the query embedding and every item in the knowledge base Return top k results
Libraries: PyMuPDF, transformers, pandas, sklearn
import fitz # PyMuPDF
def extract_text_chunks(pdf_path, chunk_size, overlap_size):
# Open the PDF file
pdf_document = fitz.open(pdf_path)
chunks = []
# Iterate through each page in the PDF
for page_num in range(len(pdf_document)):
page = pdf_document[page_num]
page_text = page.get_text()
# Split the text from the current page into chunks with overlap
start = 0
while start < len(page_text):
end = start + chunk_size
chunk = page_text[start:end]
# Store the page number with the chunk
chunks.append((page_num + 1, chunk))
# Move to the next chunk with the overlap
start += chunk_size - overlap_size
return chunks
# Parameters for extraction
pdf_path = "your_file.pdf"
chunk_size = 1000 # Size of each text chunk in characters
overlap_size = 200 # Overlap size in characters
text_chunks = extract_text_chunks(pdf_path, chunk_size, overlap_size)
# Display the chunks with page numbers
for i, (page_number, chunk) in enumerate(text_chunks):
print(f"Chunk {i + 1} (Page {page_number}):\\n{chunk}\\n{'-' * 50}")
Over the past year, I’ve helped almost 100 businesses and individuals build AI projects. By far, the most common project people ask about is a document question-answering system. Building on the previous project, we can implement this in a straightforward way.
Perform a search over the knowledge base (like the one created in Project 4) Combine user query with top k search results and pass them to a multimodal model. Create a simple Gradio user interface for the QA system.
Libraries: PyMuPDF, transformers, pandas, sklearn, together/openai, Gradio
import gradio as gr
import time
def generate_response(message, history):
""" Your code for generating a response """
return response
demo = gr.ChatInterface(
fn=generate_response,
examples=[{"text": "Hello", "files": []}],
title="Echo Bot",
multimodal=True
)
demo.launch()