## 1. pipeline ### 1.1 安装 ~~~ pip3 install ScrapyElasticSearch ~~~ ### 1.2 配置 注意版本：python3.x 否则导致错误 ~~~ ITEM_PIPELINES = { 'scrapyelasticsearch.scrapyelasticsearch.ElasticSearchPipeline': 1 } ELASTICSEARCH_SERVERS = 'http://192.168.56.120' ELASTICSEARCH_PORT = '9200' ELASTICSEARCH_INDEX = 'foodmedicine' ELASTICSEARCH_TYPE = 'forum' ELASTICSEARCH_UNIQ_KEY = 'title' ~~~ ### 1.3 item 注意：id必须有 ~~~ class Yaoq(scrapy.Item): id = scrapy.Field() title = scrapy.Field() pubDate = scrapy.Field() author = scrapy.Field() authorLocation = scrapy.Field() content = scrapy.Field() ~~~ 他是以bulk的方式向es插入数据的，所以只有数量够了才进行插入数据(500个条数一插入)