Yihua Huang
|
1491033534
|
Merge pull request #377 from jerry-sc/monitor-bug
fix the monitor bug which the spider will terminate when a seed url with port
|
2016-11-19 13:01:30 +08:00 |
|
Yihua Huang
|
228911b58c
|
Merge pull request #370 from gyk001/master
fixed #301 修复使用注解抽取JSON数据的问题
|
2016-11-19 12:59:48 +08:00 |
|
yihua.huang
|
507556d0aa
|
fix test: ProxyTest.testProxy() do not load exist proxy config
|
2016-11-19 12:54:39 +08:00 |
|
yihua.huang
|
55f131e5ef
|
#380 update fastjson to 1.2.21
|
2016-11-19 12:53:59 +08:00 |
Jerry
|
e56b8c3efc
|
fix the monitor bug which the spider will terminate when a seed url with port
|
2016-09-22 22:36:18 +08:00 |
郭玉昆
|
700898fe8a
|
fixed #301 修复使用注解抽取JSON数据的问题
|
2016-08-29 17:07:37 +08:00 |
|
Yihua Huang
|
e22d6426fc
|
Merge pull request #343 from Salon-sai/master
add: redis scheduler with priority
|
2016-08-18 13:28:25 +08:00 |
Salon.sai
|
f89a6a6826
|
add: redis scheduler with priority
|
2016-07-05 16:29:01 +08:00 |
|
yihua.huang
|
448e528140
|
update StringUtils to apache lang3 #314
|
2016-05-24 13:33:17 +08:00 |
|
yihua.huang
|
3e33959b7a
|
#319 fix javadoc
|
2016-05-24 13:17:35 +08:00 |
|
yihua.huang
|
3a6e246350
|
Merge branch 'kapsterio-fix'
|
2016-05-08 20:54:57 +08:00 |
|
yihua.huang
|
8730e3e97a
|
Merge branch 'fix' of git://github.com/kapsterio/webmagic into kapsterio-fix
|
2016-05-08 20:46:22 +08:00 |
|
Yihua Huang
|
37cb43b667
|
Merge pull request #176 from lavenderx/master
add PhantomJSDownloader
|
2016-05-08 20:36:17 +08:00 |
|
yihua.huang
|
2400ff7e1a
|
resovle conflict
|
2016-05-08 20:31:43 +08:00 |
|
yihua.huang
|
9de64ea0f2
|
Merge branch 'hepan-master'
|
2016-05-08 20:27:53 +08:00 |
|
yihua.huang
|
b7f3c4bba0
|
Merge branch 'master' of git://github.com/hepan/webmagic into hepan-master
|
2016-05-08 20:27:47 +08:00 |
|
yihua.huang
|
d8f978fd20
|
fix test in JsonPathSelectorTest #289
|
2016-05-08 19:32:03 +08:00 |
|
yihua.huang
|
25821c3c06
|
Merge branch 'EdwardsBean-proxy-strategy'
|
2016-05-08 17:53:25 +08:00 |
|
yihua.huang
|
61c28a0130
|
refactor on proxypool
|
2016-05-08 17:53:15 +08:00 |
|
yihua.huang
|
b871b210c5
|
Merge branch 'proxy-strategy' of github.com:EdwardsBean/webmagic into EdwardsBean-proxy-strategy
|
2016-05-08 17:53:02 +08:00 |
|
yihua.huang
|
b5413368de
|
update ut
|
2016-05-08 16:23:41 +08:00 |
|
yihua.huang
|
a8c9005381
|
Merge branch 'oxf1-master'
|
2016-05-08 16:18:29 +08:00 |
Jon
|
83c27ebbc4
|
增加IP代理认证功能
|
2016-05-08 16:17:58 +08:00 |
|
yihua.huang
|
ca072c5575
|
fix URL regex in GithubRepoPageProcessor #305
|
2016-05-08 12:09:45 +08:00 |
hepan
|
89c6e52863
|
代理增加用户名密码认证
|
2016-04-13 15:16:57 +08:00 |
Linker Lin
|
047cb8ff8f
|
updated versions to 0.5.4-SNAPSHOT
|
2016-04-01 14:51:59 +08:00 |
Linker Lin
|
b9544424f2
|
Update pom.xml
回滚。。。
我想，主要的问题还是主版本号没了 SNAPSHOT
|
2016-04-01 14:36:15 +08:00 |
|
Linker Lin
|
b05d605c4b
|
Update pom.xml
在纯净的环境中，需要做上面的改动。
|
2016-04-01 14:19:28 +08:00 |
zhangheng09
|
6b179c3d55
|
这个改动的原因基于两点：1）代理归还给代理池的时机应该是执行完http请求后就要尽早归还 2)http代理应该是HttpClientDownloader该考虑的事，不应该有Spider来处理，Spider并不知道它的downloader是个HttpClientDownloader
|
2016-03-12 20:09:41 +08:00 |
|
zhangheng09
|
5f106c9c69
|
当page为null时，意味着非正常的响应状态，应该抛出异常，否则SpiderListener的onSuccess方法和onError方法都会执行
|
2016-03-12 20:03:27 +08:00 |
|
yihua.huang
|
c0b8e8f8ae
|
remove .classpath .project
|
2016-01-22 14:58:22 +08:00 |
|
yihua.huang
|
ce89ebb2b4
|
remove en_doc zh_docs dir
|
2016-01-22 10:49:36 +08:00 |
|
yihua.huang
|
b9ac76aa8b
|
update readme
|
2016-01-22 10:48:44 +08:00 |
|
yihua.huang
|
a0c74a6a26
|
readme in zh
|
2016-01-22 10:47:10 +08:00 |
|
yihua.huang
|
cb61820d31
|
update spiderman link
|
2016-01-22 10:24:30 +08:00 |
|
yihua.huang
|
a8e6de4b90
|
Merge branch 'master' of git.oschina.net:flashsword20/webmagic
|
2016-01-22 10:16:58 +08:00 |
|
yihua.huang
|
cd08b230b8
|
update version to 0.5.3
|
2016-01-22 09:44:54 +08:00 |
|
yihua.huang
|
0fd4623f0a
|
Merge branch 'osc'
|
2016-01-21 19:33:30 +08:00 |
|
yihua.huang
|
ce5495ecd5
|
remove useless files
|
2016-01-21 19:31:50 +08:00 |
|
yihua.huang
|
8265c7dade
|
remove submodules for relase
|
2016-01-21 19:25:13 +08:00 |
|
yihua.huang
|
972ccd3d56
|
update version to 0.5.3-SNAPSHOT
|
2016-01-21 19:03:07 +08:00 |
|
yihua.huang
|
7edfa26f90
|
complete javadoc
|
2016-01-21 18:34:07 +08:00 |
|
yihua.huang
|
8b90b91e33
|
complete some javadoc
|
2016-01-21 18:14:10 +08:00 |
|
yihua.huang
|
2b556cf053
|
update verison to 0.5.3-SNAPSHOT
|
2016-01-21 18:05:56 +08:00 |
|
yihua.huang
|
9c5716a543
|
complete javadoc
|
2016-01-21 18:05:12 +08:00 |
|
yihua.huang
|
db3cbf6ca5
|
update version to 0.5.3-SNAPSHOT
|
2016-01-21 17:58:36 +08:00 |
|
yihua.huang
|
81ce1ffc5f
|
fix ignore
|
2016-01-21 12:36:49 +08:00 |
|
yihua.huang
|
93764fa2c9
|
ignore some test
|
2016-01-21 12:28:32 +08:00 |
|
yihua.huang
|
5706bb90af
|
update xsoup to 0.3.1
|
2016-01-20 12:59:11 +08:00 |
|
yihua.huang
|
7586e3d75c
|
add some test for github repo downloader
|
2016-01-19 08:05:53 +08:00 |