I want to add item class within an item class

Question

Final JSON will be :

            "address": ----,
            "state": ----,
            year: {
                "first": ----,
                "second": {
                    "basic": ----,
                    "Information": ----,
                    }
            },

I want to create my items.py like (just example):

class Item(scrapy.Item): 
  address = scrapy.Field()
  state = scrapy.Field()
  year = scrapy.Field(first), scrapy.Field(second)

class first(scrapy.Item):
  amounts = scrapy.Field()

class second(scrapy.Item):
  basic = scrapy.Field()
  information = scrapy.Field()

How to implement this , already checked this https://doc.scrapy.org/en/latest/topics/items.html#extending-items

how to implement nested item in scrapy?

but there are no clue about this concept ... any suggestion?

score 1 · Answer 1 · answered Jan 18 '19 at 11:58

class Item(scrapy.Item): 
  address = scrapy.Field()
  state = scrapy.Field()
  year = scrapy.Field(serializer=dict)

class Year(scrapy.Item):
  first = scrapy.Field(serializer=dict)
  second = scrapy.Field(serializer=dict)

class first(scrapy.Item):
  amounts = scrapy.Field()

class second(scrapy.Item):
  basic = scrapy.Field()
  information = scrapy.Field()

This way you can do this:

>>> b = second(basic="hello", information="hello world")
>>> a = first(amounts=3)
>>> year = Year(first=a, second=b)
>>> year
{'first': {'amounts': 3},
 'second': {'basic': 'hello', 'information': 'hello world'}}
>>> item = Item(address='address value', state='state value', year=year)
>>> item
{'address': 'address value',
 'state': 'state value',
 'year': {'first': {'amounts': 3}, 'second': {'basic': 'hello', 'information': 'hello world'}}}

score 0 · Answer 2 · answered Jan 18 '19 at 06:25

0

class Item(scrapy.Item): 
  address = scrapy.Field()
  state = scrapy.Field()
  year = scrapy.Field(first), scrapy.Field(second)    #You dont need to do like this

class first(scrapy.Item):
  amounts = scrapy.Field()     #this work and below

class second(scrapy.Item):    #and yes this work, you can do it in spider level or pipelines, just make your desired data, and pas it to year variable as you want. it will accumulate that
  basic = scrapy.Field()
  information = scrapy.Field()

Let me give you example,

first = {'first': first}
second = {'basic': basic, 'info': info}

year = {'first': first, 'second': second}

answered Jan 18 '19 at 06:25

ThunderMind

789
5
14

Thanks @thundermind, at spider level, i am able to do it very easily, but i want to declare all json elements at item level, is there any concept for that ? – helpdoc Jan 18 '19 at 06:29
There are ItemLoaders`https://doc.scrapy.org/en/latest/topics/loaders.html` for that and you can also try pipelines`https://doc.scrapy.org/en/latest/topics/item-pipeline.html`. – ThunderMind Jan 18 '19 at 09:07

I want to add item class within an item class

2 Answers2

Linked