Shape-first profiling
Summarizes which field paths exist across records before an importer, mapping or contract is written.
Day 038 / APIs / Data Profiling
Data Shape Profiler
/ Overview
A dependency-free Python CLI that profiles field presence, missing and null counts, and observed JSON types without printing source values.
Before writing an importer or validator, a developer needs to know which field paths occur, where values are absent or null, and whether a field changes type between records. Printing the underlying values can expose data that is unnecessary for that structural review.
/ Provided servicesClear thinking.
Clear thinking.
Practical delivery.
Value-free reporting
Outputs paths and aggregate counts only, so source strings, numbers and identifiers never appear in the report.
Explicit processing limits
Rejects non-object records and enforces configurable record, field-path and nesting-depth boundaries.
/ Solution
Data Shape Profiler reads a bounded local JSON array of object records, recursively inventories nested object paths, treats arrays as a type boundary, and reports presence, missing, null and observed-type counts. It emits readable text or JSON and deliberately omits every source value.
/ Impact
Nested field-path inventory
Presence, missing and null counts
Observed JSON type summaries
Arrays treated as a type boundary
Bounded records, fields and depth
Twelve standard-library tests included
Python standard library
Argparse, collections, pathlib and JSON provide the complete dependency-free command-line workflow.
Defensive JSON traversal
Recurses through objects, keeps arrays as a clear boundary and distinguishes booleans from integers.
Unittest
Twelve tests cover counts, nested paths, type distinctions, limits, value omission and CLI JSON output.