web-scraping, regex and iteration in python -


i have following url 'http://www.alriyadh.com/file/278?&page=1' write regex access urls page=2 till page=12

for example, url needed 'http://www.alriyadh.com/file/278?&page=4', not page = 14

i reckon work function iterate specified 10 pages access urls within them. have tried regex not work '.*?=[2-9]'

my aim content urls using newspaper package. want data research

thanks in advance

does not require regex, simple preset loop do.

import requests bs4 import beautifulsoup bs  url = 'http://www.alriyadh.com/file/278?&page='  page in range(2,13):     html = requests.get(url+str(page)).text     soup = bs(html)

Comments

Post a Comment

Popular posts from this blog

timeout - Handshake_timeout on RabbitMQ using python and pika from remote vm -

i have several vm's around world, identical , same python code (and same library version of pika 0.9.13). 1 of them failing because of handshake_timeout: =error report==== 17-jun-2015::12:27:07 === closing amqp connection <0.290.0> (yy.yy.yy.yy:47232 -> xx.xx.xx.xx:5672): {handshake_timeout,handshake} that taken rabbitmq log, in client code see like error:pika.adapters.base_connection:connection xx.xx.xx.xx:5672 failed: timeout unhandled exception in thread started <function listen_for_start_download_message @ 0x1e5bcf8> i have tested connection telnet following guide: http://rubybunny.info/articles/troubleshooting.html i increased handshake_timeout 40000 milliseconds , ssl_handshake_timeout 20000 milliseconds , error persists, ping rabbitmq server machine little bit higher in other machines nothing abnormal (136ms). has found similar problem sometime or can recommend way test rabbitmq connection? i'm not sure if solution problem in case pr...
Read more

gcc - MinGW's ld cannot perform PE operations on non PE output file -

i know there other similar questions out there, stackoverflow or not. i've researched lot this, , still didn't find single solution. i'm doing operative system side project. i've been doing in assembly, wanna join c code. test, made assembly code file (called test.asm): [bits 32] global _a section .text _a: jmp $ then made c file (called main.c): extern void a(void); int main(void) { a(); } to link, used file (called make.bat): "c:\mingw\bin\gcc.exe" -ffreestanding -c -o c.o main.c nasm -f coff -o asm.o test.asm "c:\mingw\bin\ld.exe" -ttext 0x100000 --oformat binary -o out.bin c.o asm.o pause i've been researching ages, , i'm still struggling find answer. hope won't flagged duplicate. acknowledge existence of similar questions, have different answers, , none work me. question: what doing wrong? old mingw versions had problem "ld" not able create non-pe files @ all. maybe current versions...
Read more

c# - Search and Add Comment with OpenXML for Word -

i'm newbie openxml sdk. i need perform following tasks on word docx on server side, , believe openxml sdk first place should into. so requirements follows: look specific text in document add comment in location of found text. is possible openxml sdk? tried go through docs, , found way search , replace text, couldn't find way add comment in appropriate location. thanks! the link ron gave indicate how insert comment. insert want, can find paragraph like paragraph p = body.descendants<paragraph>().where(p.innertext.contains(searchstring)).firstordefault(); if (p != null) { // here follow link ron post create comment // insert comment p p.insertafter() said in link } maybe have adapt bit depending matching criteria
Read more