爬取百度新闻的文字

alltolove

1. import re
   
2. import urllib.request as a
   
3. url = 'http://news.baidu.com/'
   
4. onePiece = a.urlopen(url)
   
5. a1 = onePiece.read().decode('GBK')
   
6. index = re.findall(r'[^a-z,>,<,//,=,&,",),(,A-Z,},{,+,.,/., ,]',str(a1))#取出除了所有英文字母外的东西，存成列表
   
7. index1 = (''.join(index)).strip(' ')#将列表转成字符串
   
8. with open('g:\\a.txt','w') as article:#写文件
   
9.     article.write(index1)

复制代码

零度非安全

沙发，很强势

alltolove

我又给改进了下，封装成函数然后加了个目录

1. import os
   
2. 
   
3. import re
   
4. import urllib.request as a
   
5. def searchArticle(url):
   
6.     onePiece = a.urlopen(url)
   
7.     a1 = onePiece.read().decode('GBK')
   
8.     index = re.findall(r'[^a-z,0-9,>,<,//,=,&,",),(,A-Z,},{,+,.,/., ,/s]',str(a1))#取出除了所有英文字母外的东西，存成列表
   
9.     index1 = (''.join(index)).strip(' ')#将列表转成字符串
   
10.     if not os.path.exists('G:\\article'):
    
11.         os.mkdir('G:\\article')
    
12.     with open('g:\\article\\a.txt','w') as article:#写文件
    
13.         article.write(index1)
    
14. if __name__=='__main__':
    
15.     searchArticle('http://news.baidu.com/')

复制代码

jmllljgs

我用的是python3.6.1，报错了啊

——————————————————————————

1. Traceback (most recent call last):
   
2.   File "H:/00JIMIN/练习/爬取百度文字.py", line 15, in <module>
   
3.     searchArticle('http://news.baidu.com/')
   
4.   File "H:/00JIMIN/练习/爬取百度文字.py", line 7, in searchArticle
   
5.     a1 = onePiece.read().decode('GBK')
   
6. UnicodeDecodeError: 'gbk' codec can't decode byte 0x80 in position 272: illegal multibyte sequence
   
7. >>>

复制代码

alltolove

jmllljgs 发表于 2017-7-4 17:04
我用的是python3.6.1，报错了啊
——————————————————————————

解码的问题，那个网站可能改版了