Data Structures for Petroleum Data

Petroleum engineering generates enormous volumes of data across every phase of a well's life. Drilling reports, well logs, production records, fluid analyses, pressure tests, seismic surveys: each arrives in a different format, at a different frequency, with a different structure. An engineer who cannot organize, access, and manipulate this data efficiently loses most of the working day to fighting spreadsheets instead of solving problems.

Python provides four core data structures (lists, tuples, dictionaries, and sets), each suited to a different kind of petroleum data. This chapter teaches all four through real engineering use cases, then moves to the file formats you will encounter in every petroleum engineering workflow: CSV, JSON, and LAS.

A list is an ordered, changeable sequence of values. In petroleum engineering, lists naturally represent anything that comes in a sequence: monthly production rates, depth intervals, casing diameters, or a series of pressure readings.

Lists are zero-indexed; the first element is at position 0. This is a universal convention in programming, and while it takes some adjustment if you are coming from Excel (where row 1 is the first row), it becomes natural quickly.

Production data grows over time. New months arrive, wells come online, records get corrected. Lists are mutable; you can add, remove, and change elements.

Water cut, the fraction of produced liquid that is water, is one of the most important surveillance metrics in production engineering. A rising water cut means the reservoir's water is encroaching on the oil zone. Here is how to compute and visualize it from two lists.

This is a standard production surveillance plot, the kind that appears in every monthly production report. The crossing of the oil and water curves is a significant event that reservoir engineers watch closely. In this well, it happens between months 10 and 11: the water rate overtakes the oil rate at month 11, when water cut first passes 50%.

A tuple is like a list, but it cannot be modified after creation. That sounds like a limitation, but it is actually a safety feature. Some data should never change: well coordinates, datum elevations, fundamental constants, and calibration values.

If you need a collection that cannot be accidentally modified during processing (coordinates, physical constants, calibration data) use a tuple. If you need a collection that grows, shrinks, or gets updated (production data, well lists, calculation results) use a list.

A dictionary stores key-value pairs. In petroleum engineering, this maps directly to the concept of a well header: a structured record where each field has a name and a value.

A real field contains multiple wells, each with its own header. A dictionary of dictionaries is a natural way to represent this.

Two databases should list the same wells, but do they? Which wells were drilled but never made it into the production system? Which formations show up across more than one field? Questions like these are about membership and uniqueness, and the tool built for them is the set: an unordered collection of unique values.

OD-004 and OD-006 were drilled but are not in the production database; they might be abandoned, waiting on completion, or simply missing from the records. OD-008 is producing but has no drilling record, a data quality issue that needs investigation. These are exactly the kinds of discrepancies that a petroleum data engineer encounters daily, and sets make them trivial to find.

Everything so far has lived in memory; the moment Python exits, it is gone. Real petroleum data arrives in files and has to be written back to them, and three formats cover almost everything you will touch. Each matches a different shape of data (flat tables, nested records, depth-indexed logs), and choosing the wrong one turns a one-line read into an afternoon of parsing.

Production data almost always reaches you as CSV, comma-separated values, the format SCADA systems, databases, and regulatory agencies all export to. Reading and writing it cleanly is step one of nearly every analysis in this book.

When data has structure (a well with nested completion intervals, a configuration with options inside options) CSV's flat grid stops working. JSON (JavaScript Object Notation) handles the nesting, and it maps directly to Python dictionaries, which is why API responses and metadata almost always arrive in it.

LAS (Log ASCII Standard) is the petroleum industry's standard format for well log data. Every well log you will ever work with (gamma ray, resistivity, density, neutron, sonic) is stored or exchanged as a LAS file. The code below creates one from scratch so you can see its structure: header sections (~VERSION, ~WELL, ~CURVE) followed by columnar ASCII data.

This is a triple-combo log, the everyday display a petrophysicist works from. The three tracks together tell you whether a rock interval holds hydrocarbons, how porous it is, and how thick the pay zone is. Watch the density and neutron curves over the pay zone: the neutron porosity dips below the porosity the density implies, and that crossover is the classic signature of light hydrocarbons in the rock. We compute quantitative answers from these curves in Chapter 7 (Well Log Analysis).

You will constantly need to build one collection from another: a water cut from each month's rates, the names of every well above a rate threshold, a lookup table keyed by well. Written as a loop, that is four lines of boilerplate; written as a comprehension, it is a single readable expression.

Comprehensions are not just shorter; they are more readable when the transformation is simple. When the logic inside gets complex (multiple conditions, nested calculations), a regular loop with a named function is the clearer choice.

This chapter covered the data structures that underpin all petroleum data processing:

• Lists store ordered sequences: production histories, depth arrays, casing programs. They are mutable and indexed from zero.
• Tuples store fixed data: well coordinates, casing specifications, reference constants. Their immutability is a safety feature, not a limitation.
• Dictionaries store key-value pairs: well headers, completion records, field databases. Nested dictionaries model the hierarchical structure of petroleum data naturally.
• Sets store unique values: well inventories, formation lists, data reconciliation. Set operations (intersection, difference, union) answer data quality questions instantly.
• CSV files hold tabular production and engineering data. Python's csv module reads and writes them with minimal code.
• JSON files hold structured metadata and configuration. They map directly to Python dictionaries.
• LAS files hold well log data, the petroleum industry's standard since the CWLS first published the Log ASCII Standard in 1990. The columnar format contains depth-indexed measurements from wireline and LWD tools.
• Comprehensions provide concise, readable data transformations for lists and dictionaries.

In the next chapter, we move from Python's built-in structures to the libraries that make petroleum data science practical: NumPy for numerical computation and Pandas for tabular data analysis at scale.