yihua.huang
|
aaccc93215
|
new version
|
2017-01-21 12:04:12 +08:00 |
|
yihua.huang
|
3e633c6871
|
version
|
2017-01-21 11:51:14 +08:00 |
|
yihua.huang
|
f45e2f118b
|
for release
|
2017-01-21 11:38:36 +08:00 |
|
yihua.huang
|
d60615f503
|
修复使用startUrls没有设置domain导致使用cookie空指针的问题#438
|
2017-01-21 11:29:42 +08:00 |
|
yihua.huang
|
407fbb6130
|
refactor logger#445
|
2017-01-21 11:05:54 +08:00 |
|
Yihua Huang
|
f29a10472f
|
Merge pull request #414 from jsbd/master
新增构造函数，支持crawl.js路径自定义，因为当其他项目依赖此jar包时，runtime.exec()执行phantomjs命令时无使用法jar包中的crawl.js
|
2017-01-21 10:50:48 +08:00 |
|
Yihua Huang
|
93e7040fe5
|
Merge pull request #445 from ckex/develop
optimize code.
|
2017-01-21 10:45:32 +08:00 |
Ckex.zha
|
0dc26c8ca0
|
optimize code.
|
2017-01-20 14:03:26 +08:00 |
|
Yihua Huang
|
4f76d62d4f
|
Merge pull request #444 from ckex/develop
绕过安全证书
|
2017-01-18 23:42:51 +08:00 |
|
Ckex.zha
|
e4af05a6f2
|
绕过安全证书
|
2017-01-18 17:28:01 +08:00 |
|
Yihua Huang
|
e919bbd858
|
Merge pull request #443 from xbynet/master
解决post/redirect/post 302跳转问题
|
2017-01-17 23:20:37 +08:00 |
xbynet@outlook.com
|
c23627bf63
|
解决post/redirect/post 302跳转问题
|
2017-01-17 00:07:01 +08:00 |
|
yihua.huang
|
7ca4ed04b4
|
add Gather Platform to readme zh
|
2017-01-14 06:25:39 +08:00 |
Jsbd
|
6d78d51fc0
|
Merge branch 'master' into master
|
2016-12-27 14:15:40 +08:00 |
|
yihua.huang
|
e3131af856
|
remove user manual link
|
2016-12-24 07:34:25 +08:00 |
|
yihua.huang
|
7f0d79ccb0
|
readme
|
2016-12-19 09:38:33 +08:00 |
|
yihua.huang
|
ab1dad3a71
|
remove lib
|
2016-12-18 11:49:20 +08:00 |
|
yihua.huang
|
a6b2e307f3
|
docs for 0.6.0
|
2016-12-18 11:48:57 +08:00 |
|
yihua.huang
|
d69204b919
|
0.6.0
|
2016-12-18 11:45:43 +08:00 |
|
yihua.huang
|
9bdb48b2d0
|
version 0.6.0
|
2016-12-18 11:20:28 +08:00 |
|
yihua.huang
|
eeb607fd0e
|
将Spider.processRequest()抛出异常改回原来的逻辑
|
2016-12-18 11:04:58 +08:00 |
|
yihua.huang
|
97592d6720
|
Version 0.6.0
|
2016-12-18 10:58:24 +08:00 |
|
yihua.huang
|
42e5e623b4
|
remove project avalon
|
2016-12-18 10:52:01 +08:00 |
|
yihua.huang
|
00dfebbceb
|
#424 remove guava dep and add fix docs
|
2016-12-18 10:45:50 +08:00 |
|
yihua.huang
|
c2531c6817
|
clean dependency
|
2016-12-18 08:34:46 +08:00 |
|
yihua.huang
|
a960a39c44
|
fix compile error for example change
|
2016-12-18 08:32:14 +08:00 |
|
yihua.huang
|
a3ee9e3d08
|
fix example
|
2016-12-18 08:18:26 +08:00 |
|
yihua.huang
|
7476ceccee
|
more stable test
|
2016-12-18 08:15:26 +08:00 |
|
yihua.huang
|
5ce3fdfe5a
|
some refactor in log
|
2016-12-18 08:15:09 +08:00 |
|
yihua.huang
|
98163a3e40
|
update examples
|
2016-12-18 07:46:18 +08:00 |
|
yihua.huang
|
243ebc22fa
|
#374 update httpclient version to 4.5.2
|
2016-12-18 07:15:40 +08:00 |
|
yihua.huang
|
b090dcd20d
|
sepcific error page for HttpClientDownloaderTest to avoid test error when local port is available
|
2016-12-18 07:15:06 +08:00 |
|
yihua.huang
|
8f942d6fe2
|
#419 修复抓取https链接线程无法结束导致进程一直运行的问题
|
2016-12-18 06:56:01 +08:00 |
|
Jsbd
|
1b886d48a2
|
新增PhantomJSDownloader构造函数，支持crawl.js路径自定义，因为当其他项目依赖此jar包时，runtime.exec()执行phantomjs命令时无使用法jar包中的crawl.js
|
2016-12-08 14:29:42 +08:00 |
|
Jsbd
|
d1f2e65e5d
|
新增PhantomJSDownloader构造函数，支持crawl.js路径自定义，因为当其他项目依赖此jar包时，runtime.exec()执行phantomjs命令时无使用法jar包中的crawl.js
|
2016-12-08 14:28:48 +08:00 |
|
Jsbd
|
f8a2328ead
|
Merge pull request #1 from code4craft/master
merge
|
2016-12-02 17:22:55 +08:00 |
|
Yihua Huang
|
1987cd3ae1
|
Merge pull request #408 from code4craft/0.6.0
groovy demo
|
2016-12-02 14:20:46 +08:00 |
|
Yihua Huang
|
65fe2c4487
|
Merge pull request #407 from jsbd/master
为PhantomJSDownloader添加新的构造函数，支持phantomjs自定义命令
|
2016-12-02 14:05:58 +08:00 |
|
Jsbd
|
ebc61363c8
|
为PhantomJSDownloader添加新的构造函数，支持phantomjs自定义命令
为PhantomJSDownloader添加新的构造函数，支持phantomjs自定义命令
example:
* phantomjs.exe 支持windows环境
* phantomjs --ignore-ssl-errors=yes 忽略抓取地址是https时的一些错误
* /usr/local/bin/phantomjs 命令的绝对路径，避免因系统环境变量引起的IOException
|
2016-12-02 10:17:46 +08:00 |
|
yihua.huang
|
fdf39eb99d
|
open in new page
|
2016-11-25 18:20:59 +08:00 |
|
yihua.huang
|
1e74494708
|
add related link
|
2016-11-25 18:19:32 +08:00 |
|
yihua.huang
|
b92e6b04f0
|
#400 修复FileCacheQueueScheduler自己设置DuplicateRemover会导致NPE的问题
|
2016-11-25 08:30:24 +08:00 |
|
yihua.huang
|
dafd2b77ff
|
fix GithubRepoPageProcessor in example
|
2016-11-24 08:18:06 +08:00 |
|
yihua.huang
|
cfed860fb9
|
Merge branch 'master' of github.com:code4craft/webmagic
|
2016-11-22 17:00:27 +08:00 |
|
yihua.huang
|
2189aab652
|
fix test
|
2016-11-22 16:58:49 +08:00 |
|
Yihua Huang
|
1491033534
|
Merge pull request #377 from jerry-sc/monitor-bug
fix the monitor bug which the spider will terminate when a seed url with port
|
2016-11-19 13:01:30 +08:00 |
|
Yihua Huang
|
228911b58c
|
Merge pull request #370 from gyk001/master
fixed #301 修复使用注解抽取JSON数据的问题
|
2016-11-19 12:59:48 +08:00 |
|
yihua.huang
|
507556d0aa
|
fix test: ProxyTest.testProxy() do not load exist proxy config
|
2016-11-19 12:54:39 +08:00 |
|
yihua.huang
|
55f131e5ef
|
#380 update fastjson to 1.2.21
|
2016-11-19 12:53:59 +08:00 |
Jerry
|
e56b8c3efc
|
fix the monitor bug which the spider will terminate when a seed url with port
|
2016-09-22 22:36:18 +08:00 |