soul
|
bc828e1384
|
修复formatter初始化未传参bug
|
2017-05-25 12:17:10 +08:00 |
yihua.huang
|
a8c2e6c729
|
alpha release
|
2017-05-20 12:51:16 +08:00 |
|
yihua.huang
|
3c1338193b
|
for 0.7.0.alpha
|
2017-05-20 12:34:09 +08:00 |
|
yihua.huang
|
e8abc28072
|
#552 add some log when crawler stop
|
2017-05-20 11:44:51 +08:00 |
zhuyue
|
9e1b7ed3f7
|
Update RegexSelector.java
|
2017-05-05 10:47:10 +08:00 |
|
zhuyue
|
975adf7072
|
Merge pull request #2 from zhuyuesut/zhuyuesut-fixbug
Update RegexSelectorTest.java
|
2017-05-03 18:36:21 +08:00 |
|
zhuyue
|
c80f25edbd
|
Update RegexSelectorTest.java
简单的增加了一点测试
|
2017-05-03 18:33:23 +08:00 |
|
zhuyue
|
39c3c2f904
|
Merge pull request #1 from zhuyuesut/zhuyuesut-fixbug
Update RegexSelector.java
|
2017-05-03 18:28:05 +08:00 |
|
zhuyue
|
0c359a2bde
|
Update RegexSelector.java
|
2017-05-03 18:24:41 +08:00 |
|
zhuyue
|
c3183252ac
|
Update RegexSelector.java
|
2017-05-03 18:24:19 +08:00 |
|
yihua.huang
|
cbf80af5dd
|
test for SimpleProxyProvider #535
|
2017-04-16 10:50:27 +08:00 |
|
yihua.huang
|
eb632a93d3
|
SimpleProxyProvider #535
|
2017-04-16 10:43:56 +08:00 |
|
yihua.huang
|
62a6985103
|
update javadoc config
|
2017-04-15 20:07:13 +08:00 |
|
yihua.huang
|
d38d51dfcb
|
fix javadoc
|
2017-04-15 12:24:50 +08:00 |
|
Yihua Huang
|
0cd2f6031a
|
Merge pull request #528 from GZhY/master
对 WebMagic-0.7.0 的小修补
|
2017-04-10 07:43:53 +08:00 |
GZhY
|
5f34adf938
|
完善 LinksSelector.selectList 的测试用例
|
2017-04-09 21:29:01 +08:00 |
|
GZhY
|
ce3f0ac239
|
删除 fixAllRelativeHrefs 并修复 SeleniumDownloader 对 fixAllRelativeHrefs 的依赖
|
2017-04-09 21:01:32 +08:00 |
|
GZhY
|
bc6e81e00f
|
修复checkElementAndConvert方法注释中注释错误
|
2017-04-09 20:40:00 +08:00 |
|
yihua.huang
|
4a2c0f4f97
|
add returnProxy for proxyProvider
|
2017-04-09 09:28:36 +08:00 |
|
yihua.huang
|
1b04a7f2b3
|
#527 move logic check from downloaderto spider
|
2017-04-09 09:23:10 +08:00 |
|
Yihua Huang
|
6ead04a758
|
Merge pull request #524 from code4craft/proxyRefactor
HttpClient部分重构
|
2017-04-08 23:19:08 +08:00 |
|
yihua.huang
|
0f4d6e8b12
|
#525 remove port in UrlUtils.getDomain()
|
2017-04-08 23:17:00 +08:00 |
|
yihua.huang
|
a1ae632b62
|
test for request cookies and headers
|
2017-04-08 23:13:16 +08:00 |
|
yihua.huang
|
db67db8103
|
#523 remove fixAllRelativeHrefs by default, get absolute urls for links()
|
2017-04-08 22:06:18 +08:00 |
|
yihua.huang
|
abd020b45b
|
some comments
|
2017-04-08 20:16:17 +08:00 |
|
yihua.huang
|
2622b448b8
|
fix test
|
2017-04-08 20:09:43 +08:00 |
|
yihua.huang
|
b06a248c00
|
fix test
|
2017-04-08 20:06:04 +08:00 |
|
yihua.huang
|
1cfbd13aae
|
refacor in httpclientdownloader
|
2017-04-08 20:04:56 +08:00 |
|
yihua.huang
|
83ada9749e
|
fix test
|
2017-04-08 12:16:34 +08:00 |
|
yihua.huang
|
fe95a6842f
|
Request再次重构:去掉params，仅保留HttpRequestBody
|
2017-04-08 12:12:39 +08:00 |
|
yihua.huang
|
395396c68e
|
增加HttpRequestBody
|
2017-04-08 11:59:52 +08:00 |
|
Yihua Huang
|
25df6650d9
|
Merge pull request #513 from xbynet/master
Request支持设置header与cookie、新增POST请求时,XML、JSON参数支持、Page支持获取响应header
|
2017-04-03 07:57:40 +08:00 |
xbynet
|
c93a8a2722
|
修复字符编码检测BUG
|
2017-03-31 18:27:18 +08:00 |
|
yihua.huang
|
74110e6ec5
|
remove useless file
|
2017-03-30 09:19:00 +08:00 |
|
yihua.huang
|
b100dfe273
|
update version
|
2017-03-30 09:18:12 +08:00 |
|
xbynet@outlook.com
|
1c24baa8d1
|
Request支持设置header与cookie
新增POST请求时,XML、JSON参数支持
Page支持获取响应header
|
2017-03-29 02:27:06 +08:00 |
xbynet
|
5824e9515b
|
Merge pull request #3 from code4craft/master
合并最新代码
|
2017-03-29 02:20:16 +08:00 |
|
yihua.huang
|
6bd197859b
|
fix test
|
2017-03-27 09:55:50 +08:00 |
|
yihua.huang
|
f23e138c72
|
add response headers to Page #508
|
2017-03-27 09:52:25 +08:00 |
|
yihua.huang
|
ba000b364c
|
remove site.addStartUrl in samples
|
2017-03-21 08:00:15 +08:00 |
|
yihua.huang
|
3824232171
|
remove site.addStartUrl in samples
|
2017-03-21 07:59:33 +08:00 |
|
yihua.huang
|
8564d51e56
|
remove site.addStartUrl in samples
|
2017-03-21 07:57:56 +08:00 |
|
yihua.huang
|
c13110c4cb
|
fix samples
|
2017-03-21 07:53:43 +08:00 |
|
yihua.huang
|
c51ac6017c
|
remove Site.addStartRequest() etc. #494
|
2017-03-21 07:32:34 +08:00 |
|
yihua.huang
|
68050fc88e
|
test pass
|
2017-03-21 07:19:12 +08:00 |
|
yihua.huang
|
474b7c9d57
|
refactor
|
2017-03-20 23:13:49 +08:00 |
|
yihua.huang
|
25c81013ca
|
new proxy pool api
|
2017-03-20 22:47:24 +08:00 |
|
yihua.huang
|
46297deaa1
|
HttpUriRequestConverter
|
2017-03-20 22:43:32 +08:00 |
|
yihua.huang
|
1d86f7c048
|
compile passed in httpclientDownloader
|
2017-03-20 22:40:14 +08:00 |
|
yihua.huang
|
b71f379512
|
fix
|
2017-03-18 12:18:00 +08:00 |