Loading and inspecting data: pandas DataFrames for order and survey data

Two files, one question. Picture Monday morning. Your manager forwards two files, last month's orders export and the customer survey, and asks one question: are we doing okay? You could open both in Excel and scroll. But scrolling hides things: a price column stored as text, a few blank ratings, a product spelled three different ways. Any of those can quietly break your answer. So how do you get a trustworthy first look at a file in a few minutes? That's what this lesson is about. We'll use pandas, a free Python library, to load the data and ask it a few quick questions before we trust a single number.

What you will learn. By the end of this lesson you'll be able to do three things. First, load a CSV file into a pandas DataFrame, which is simply a table that Python can work with. Second, inspect it with a small set of commands: head, shape, info and describe. Third, spot the warning signs that tell you the data needs cleaning, like missing values or numbers stored as text. These same few commands work on a hundred rows or ten million, so once they become a habit, every new file starts the same calm, predictable way.

What a DataFrame is. Pandas is the standard Python library for working with tables. Its main object is the DataFrame. Think of it as one Excel sheet. Rows are records, like one order each. Columns are fields, like the order date or the price. Each column holds one data type, which is just the kind of value it contains: whole numbers, decimals, text or dates. Pandas also adds an index, a label for every row, a bit like the row numbers down the left edge in Excel. Once your data is in a DataFrame, you ask questions with one line of code instead of dragging formulas around.

Loading the orders export. Here's our orders export. In real work you'd call read CSV with a file name. To keep this example runnable anywhere, I've pasted the file's text straight into the code and wrapped it in StringIO, which makes a piece of text behave like a file. Read CSV takes the header line as column names and every other line as a row. Then head of three shows the first three rows. It worked: order ten oh one is Ana buying two mugs at twelve fifty. Always look at a few rows first. It's the fastest way to catch a file that loaded with the wrong separator or shifted columns.

How big, and what types?. Next come two questions: how big is it, and what's in each column? Shape gives rows first, then columns, here six and six. Info lists every column with how many values are not missing, plus its data type. Quantity is int sixty-four, a whole number. Price is float sixty-four, a number with decimals. Date, customer and product show object, which is pandas' word for text. Notice that date is text, not a real date yet. That's fine for now, but it's exactly the kind of detail info exists to reveal. We'll fix types in the next lesson.

Summary numbers with describe. Describe gives summary statistics, short numbers that sum up a whole column. Let's read the qty column. Count is how many values there are, six. Mean is the ordinary average, about two point one seven items per order. Std, the standard deviation, measures spread: roughly how far a typical value sits from the average. Min and max are the smallest and largest. The twenty-five, fifty and seventy-five percent rows are percentiles, the value below which that share of the data falls. Fifty percent is the median, the middle value, here two items. The price column gets the same summary, with prices running from eight to twenty dollars. Is anyone ordering a strange amount? A max of four says no.

Checking the mean by hand. Let's check describe by hand, so you know where its numbers come from. Add the six quantities: two plus one plus one plus three plus two plus four equals thirteen. The mean divides by six orders: thirteen sixths. That is about two point one seven items per order, just like describe said. For the median, sort the values: one, one, two, two, three, four. With six values there are two in the middle, the median of the two middle values: two plus two, divided by two, is two items.

Inspecting the survey. Survey data looks different: a rating, the channel where the customer heard about us, and a free-text comment. For text columns like channel, value counts is your best friend. It counts each distinct value, like a quick pivot table: three from email, two from ads and one from search. Then is NA marks every missing cell, and sum counts those blanks per column. One rating is missing and two comments are empty. That missing rating matters. If you average the ratings, you're averaging five people, not six, and your report should say so.

From inspection to a first answer. Now we can give a first, honest answer. Len gives the number of rows, six survey responses. Count gives how many ratings are actually filled in, five. Mean gives the average rating, four point oh. By default pandas skips missing values when it averages, which is usually what you want, but only if you know it happened. Let's check that average by hand next.

Checking the average rating. Add the five ratings: five plus four plus five plus two plus four equals twenty. Divide by the five people who answered, not six: twenty divided by five is four. So the sentence for your manager is: the average rating is four point oh out of five, from five of six responses. That small note about the base turns a number into something people can trust.

The five-minute routine. Put together, every new file gets the same routine. Load it with read CSV. Look at a few rows with head. Check the size with shape, so you know if the export was cut off. Check types and missing values with info. Get summary numbers with describe, and use value counts for text columns. Finally, write a few notes: row count, missing values, anything stored as the wrong type. It takes about five minutes, and it saves you from presenting a chart built on broken data. It's the data version of checking your mirrors before driving.

Common mistake: numbers stored as text. Here's the classic trap. Many exports write prices with a dollar sign. Pandas can't read a dollar sign as part of a number, so the whole price column becomes object, which means text. Nothing crashes. But describe quietly leaves price out: it only summarises order id. And sum does something strange: instead of adding, it glues the texts together into one long string. If you skipped info, you might not notice until a report looks odd, or worse, never notice at all. The fix is to strip the symbol and convert the column to numbers, which is the first thing we'll do next lesson.

Recap. Let's recap. A DataFrame is a table, like one Excel sheet, where every column has a single data type. Before you calculate anything, inspect it: head to see rows, shape for size, info for types and missing values, describe for summary numbers, and value counts for text columns. And always report the base behind a number, because missing values and text-typed numbers can change or hide your results without any error message. These habits are what turn gut feel into numbers you can defend in a meeting.

Your exercise. Now it's your turn. Create a small support ticket dataset with eight rows and three columns: ticket id, channel, such as email or chat, and minutes to resolve. Leave one minutes value blank and write another one as forty-five space min, the way a messy export would. Load it with read CSV and StringIO, then run the full routine: head, shape, info, describe and value counts on channel. Finally, write a three-line inspection note: how many rows, what's missing, and which column has the wrong type. Bring that note to the next lesson, where we'll clean it.