I have a working model to receive a json
data set using pydantic
. The model data set looks like this:
data = {'thing_number': 123,
'thing_description': 'duck',
'thing_amount': 4.56}
What I would like to do is have a list of json
files as the data set and be able to validate them. Ultimately the list will be converted to records in pandas
for further processing. My goal is to validate an arbitrarily long list of json
entries that looks something like this:
bigger_data = [{'thing_number': 123,
'thing_description': 'duck',
'thing_amount': 4.56},
{'thing_number': 456,
'thing_description': 'cow',
'thing_amount': 7.89}]
The basic setup I have now is as follows. Note that adding the class ItemList
is part of the attempt to get the arbitrary length to work.
from typing import List
from pydantic import BaseModel
from pydantic.schema import schema
import json
class Item(BaseModel):
thing_number: int
thing_description: str
thing_amount: float
class ItemList(BaseModel):
each_item: List[Item]
The basic code will then produce what I think I'm looking for in an array object that will take Item
objects.
item_schema = schema([ItemList])
print(json.dumps(item_schema, indent=2))
{
"definitions": {
"Item": {
"title": "Item",
"type": "object",
"properties": {
"thing_number": {
"title": "Thing_Number",
"type": "integer"
},
"thing_description": {
"title": "Thing_Description",
"type": "string"
},
"thing_amount": {
"title": "Thing_Amount",
"type": "number"
}
},
"required": [
"thing_number",
"thing_description",
"thing_amount"
]
},
"ItemList": {
"title": "ItemList",
"type": "object",
"properties": {
"each_item": {
"title": "Each_Item",
"type": "array",
"items": {
"$ref": "#/definitions/Item"
}
}
},
"required": [
"each_item"
]
}
}
}
The setup works on a singe json item being passed:
item = Item(**data)
print(item)
Item thing_number=123 thing_description='duck' thing_amount=4.56
But when I try and pass the single item into the ItemList
model it returns an error:
item_list = ItemList(**data)
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
<ipython-input-94-48efd56e7b6c> in <module>
----> 1 item_list = ItemList(**data)
/opt/conda/lib/python3.7/site-packages/pydantic/main.cpython-37m-x86_64-linux-gnu.so in pydantic.main.BaseModel.__init__()
/opt/conda/lib/python3.7/site-packages/pydantic/main.cpython-37m-x86_64-linux-gnu.so in pydantic.main.validate_model()
ValidationError: 1 validation error for ItemList
each_item
field required (type=value_error.missing)
I've also tried passing bigger_data
into the array thinking that it would need to start as a list. that also returns an error - - Although, I at least have a better understanding of the dictionary error I can't figure out how to resolve.
item_list2 = ItemList(**data_big)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-100-8fe9a5414bd6> in <module>
----> 1 item_list2 = ItemList(**data_big)
TypeError: MetaModel object argument after ** must be a mapping, not list
Thanks.
Other Things I've Tried
I've tried passing the data into the specific key with a little more luck (maybe?).
item_list2 = ItemList(each_item=data_big)
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
<ipython-input-111-07e5c12bf8b4> in <module>
----> 1 item_list2 = ItemList(each_item=data_big)
/opt/conda/lib/python3.7/site-packages/pydantic/main.cpython-37m-x86_64-linux-gnu.so in pydantic.main.BaseModel.__init__()
/opt/conda/lib/python3.7/site-packages/pydantic/main.cpython-37m-x86_64-linux-gnu.so in pydantic.main.validate_model()
ValidationError: 6 validation errors for ItemList
each_item -> 0 -> thing_number
field required (type=value_error.missing)
each_item -> 0 -> thing_description
field required (type=value_error.missing)
each_item -> 0 -> thing_amount
field required (type=value_error.missing)
each_item -> 1 -> thing_number
field required (type=value_error.missing)
each_item -> 1 -> thing_description
field required (type=value_error.missing)
each_item -> 1 -> thing_amount
field required (type=value_error.missing)
To avoid having "each_item"
in the ItemList
, you can use the __root__
Pydantic keyword:
from typing import List
from pydantic import BaseModel
class Item(BaseModel):
thing_number: int
thing_description: str
thing_amount: float
class ItemList(BaseModel):
__root__: List[Item] # ⯇-- __root__
To build the item_list
:
just_data = [
{"thing_number": 123, "thing_description": "duck", "thing_amount": 4.56},
{"thing_number": 456, "thing_description": "cow", "thing_amount": 7.89},
]
item_list = ItemList(__root__=just_data)
a_json_duck = {"thing_number": 123, "thing_description": "duck", "thing_amount": 4.56}
item_list.__root__.append(a_json_duck)
The web-frameworks supporting Pydantic often jsonify such ItemList
as a JSON array without intermediate __root__
keyword.