![]() Speaking from this, requests.get is more explicit. Also, sleep(0.5) and sleep(5) don't make any sense. please wait a few seconds") is wrong as you got the data at response = get(url). So just use for item in json_data.Īctually, print("Getting data from site. I think you just want to get the information for each item. Never use both for and while on the same variable. If response.status_code != 200, I think the function should ~return an empty result~ raise an exception like said by use json_data a lot, you could define items = json_data instead, but see below. Using = is more appropriate than is in general, hence response.status_code = 200 Index is not useful, and range(0, n) is the same as range(n) (item, item, item, item)įor item in response.json()Ĭolumns=Ĭreating functions that read and modify global variables is not a good idea, for example if someone wants to reuse your function, they won't know about side effects. You don't have the use 4 separate lists, but can instead create one list and add the column names afterwards. I suggest to stick with for loops, Python excels at readable for loops ![]() Why use a while inside the for and return when finished with the whileĮither loop with a for or a while, not both! Because the while currently disregards the for loop. Response.raise_for_status() this will create an exception when there is an 4xx, 5xx status If response.status_code is 200: should be = instead of is You don't break or return when an invalid status is encountered Would one list be more than sufficient? Am I approaching this wrongly.Ĭonstants ( url) should be UPPER_SNAKE_CASEįunctions ( getShockingSales()) should be lower_snake_case I have many repetitive List that has data being append to them. ![]() #Webscraper request interval codeBut I've come across a common smelly code among several of my applications. ![]() At the moment, I'm learning and experimenting on the use of web scraping content from different varieties of web pages. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |