Filip/matrix-spec
1
0
Fork 0
mirror of https://github.com/matrix-org/matrix-spec synced 2026-02-04 05:13:42 +01:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

Merge pull request #2491 from matrix-org/benp/proposals-scraper-fix

Browse source 
fix pagination in proposals scraper
This commit is contained in:
Ben Parsons 2020-04-07 16:24:30 +01:00 committed by GitHub
parent 78a0843758 7c037d2490
commit 57f3c8494f
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
scripts/proposals.py
View file

@ -38,7 +38,10 @@ def getpage(url):
pagecount = 1
for link in resp.links.values():
if link['rel'] == 'last':
pagecount = int(re.search('page=(.+?)', link['url']).group(1))
# we extract the pagecount from the `page` param of the last url
# in the response, eg
# 'https://api.github.com/repositories/24998719/issues?state=all&labels=proposal&page=10'
pagecount = int(re.search('page=(\d+)', link['url']).group(1))
val = resp.json()
if not isinstance(val, list):