University of Wisconsin Write a Scraper Through Terminal Phyton Coding Task
Question Description
I’m working on a python question and need an explanation and answer to help me learn.
Create a file inside the spiders
directory called tmdb_spider.py
. Add the following lines to the file:
import scrapy
class TmdbSpider(scrapy.Spider):
name = 'tmdb_spider'
start_urls = ['https://www.themoviedb.org/tv/1411-person-of-interest/']
implement three parsing methods for the TmdbSpider
class.
parse(self, response)
should assume that you start on a movie page, and then navigate to the Cast & Crew page. Remember that this page has url<movie_url>cast
. (You are allowed to hardcode that part.) Once there, theparse_full_credits(self,response)
should be called, by specifying this method in thecallback
argument to a yieldedscrapy.Request
. Theparse()
method does not return any data. This method should be no more than 5 lines of code, excluding comments and docstrings.parse_full_credits(self, response)
should assume that you start on the Cast & Crew page. Its purpose is to yield ascrapy.Request
for the page of each actor listed on the page. Crew members are not included. The yielded request should specify the methodparse_actor_page(self, response)
should be called when the actoràpage is reached. Theparse_full_credits()
method does not return any data. This method should be no more than 5 lines of code, excluding comments and docstrings.parse_actor_page(self, response)
should assume that you start on the page of an actor. It should yield a dictionary with two key-value pairs, of the form{"actor" : actor_name, "movie_or_TV_name" : movie_or_TV_name}
. The method should yield one such dictionary for each of the movies or TV shows on which that actor has worked. Note that you will need to determine both the name of the actor and the name of each movie or TV show.
Have a similar assignment? "Place an order for your assignment and have exceptional work written by our team of experts, guaranteeing you A results."