yihua.huang
|
592fa2c0f1
|
add site header test
|
2017-06-03 11:56:18 +08:00 |
|
yihua.huang
|
07cbc9947b
|
fix pom error in webmagic-script
|
2017-06-03 11:40:40 +08:00 |
|
yihua.huang
|
19d34dbb65
|
not add bracket to regex in RegexSelector #559
|
2017-06-03 11:30:07 +08:00 |
|
Yihua Huang
|
b6b991a09b
|
Merge pull request #556 from zhuyuesut/master
增加对零宽断言的支持
|
2017-06-03 11:10:56 +08:00 |
|
yihua.huang
|
f02f469c69
|
add test #570
|
2017-06-03 11:02:31 +08:00 |
|
yihua.huang
|
bb0eb69acf
|
update ZhihuPageProcessor example
|
2017-06-03 10:42:17 +08:00 |
|
yihua.huang
|
2d693580fc
|
add test
|
2017-06-01 22:28:03 +08:00 |
|
yihua.huang
|
b879b0eed0
|
fix redisscheduler #583
|
2017-06-01 22:25:01 +08:00 |
|
yihua.huang
|
818a2b2408
|
invite kotlin experimental
|
2017-06-01 07:51:11 +08:00 |
|
yihua.huang
|
3c653d941a
|
Merge branch 'master' of github.com:code4craft/webmagic
|
2017-05-29 21:26:09 +08:00 |
|
yihua.huang
|
5a9328c32e
|
update link
|
2017-05-29 21:25:57 +08:00 |
|
Yihua Huang
|
9903f0367d
|
Merge pull request #570 from SoulZhong/master
修复formatter初始化未传参bug
|
2017-05-29 17:49:42 +08:00 |
|
yihua.huang
|
ec7c52104f
|
update logo
|
2017-05-29 15:28:08 +08:00 |
|
yihua.huang
|
2482c2c877
|
update version in readme
|
2017-05-29 15:23:47 +08:00 |
|
yihua.huang
|
858d535ec7
|
remove useless files
|
2017-05-29 15:00:22 +08:00 |
|
yihua.huang
|
2e35e149be
|
for 0.7.1
|
2017-05-29 14:41:49 +08:00 |
|
yihua.huang
|
17d8bfa907
|
docs and pgp version
|
2017-05-29 09:36:42 +08:00 |
|
yihua.huang
|
17478fcfc4
|
0.7.0 release
|
2017-05-29 09:30:56 +08:00 |
|
yihua.huang
|
636359300f
|
add Site.disableCookieManagement #577
|
2017-05-29 08:29:53 +08:00 |
|
yihua.huang
|
49de9374cd
|
new SimpleHttpClient #576
|
2017-05-27 17:30:19 +08:00 |
|
yihua.huang
|
7ffc6998ef
|
add isExtractLinks to OOSpider #575
|
2017-05-27 16:20:06 +08:00 |
|
yihua.huang
|
8999ea9320
|
add public constructor for SimpleProxyProvider
|
2017-05-27 16:09:02 +08:00 |
soul
|
bc828e1384
|
修复formatter初始化未传参bug
|
2017-05-25 12:17:10 +08:00 |
|
yihua.huang
|
a8c2e6c729
|
alpha release
|
2017-05-20 12:51:16 +08:00 |
|
yihua.huang
|
3c1338193b
|
for 0.7.0.alpha
|
2017-05-20 12:34:09 +08:00 |
|
yihua.huang
|
e8abc28072
|
#552 add some log when crawler stop
|
2017-05-20 11:44:51 +08:00 |
zhuyue
|
9e1b7ed3f7
|
Update RegexSelector.java
|
2017-05-05 10:47:10 +08:00 |
|
zhuyue
|
975adf7072
|
Merge pull request #2 from zhuyuesut/zhuyuesut-fixbug
Update RegexSelectorTest.java
|
2017-05-03 18:36:21 +08:00 |
|
zhuyue
|
c80f25edbd
|
Update RegexSelectorTest.java
简单的增加了一点测试
|
2017-05-03 18:33:23 +08:00 |
|
zhuyue
|
39c3c2f904
|
Merge pull request #1 from zhuyuesut/zhuyuesut-fixbug
Update RegexSelector.java
|
2017-05-03 18:28:05 +08:00 |
|
zhuyue
|
0c359a2bde
|
Update RegexSelector.java
|
2017-05-03 18:24:41 +08:00 |
|
zhuyue
|
c3183252ac
|
Update RegexSelector.java
|
2017-05-03 18:24:19 +08:00 |
|
yihua.huang
|
cbf80af5dd
|
test for SimpleProxyProvider #535
|
2017-04-16 10:50:27 +08:00 |
|
yihua.huang
|
eb632a93d3
|
SimpleProxyProvider #535
|
2017-04-16 10:43:56 +08:00 |
|
yihua.huang
|
62a6985103
|
update javadoc config
|
2017-04-15 20:07:13 +08:00 |
|
yihua.huang
|
d38d51dfcb
|
fix javadoc
|
2017-04-15 12:24:50 +08:00 |
|
Yihua Huang
|
0cd2f6031a
|
Merge pull request #528 from GZhY/master
对 WebMagic-0.7.0 的小修补
|
2017-04-10 07:43:53 +08:00 |
GZhY
|
5f34adf938
|
完善 LinksSelector.selectList 的测试用例
|
2017-04-09 21:29:01 +08:00 |
|
GZhY
|
ce3f0ac239
|
删除 fixAllRelativeHrefs 并修复 SeleniumDownloader 对 fixAllRelativeHrefs 的依赖
|
2017-04-09 21:01:32 +08:00 |
|
GZhY
|
bc6e81e00f
|
修复checkElementAndConvert方法注释中注释错误
|
2017-04-09 20:40:00 +08:00 |
|
yihua.huang
|
4a2c0f4f97
|
add returnProxy for proxyProvider
|
2017-04-09 09:28:36 +08:00 |
|
yihua.huang
|
1b04a7f2b3
|
#527 move logic check from downloaderto spider
|
2017-04-09 09:23:10 +08:00 |
|
Yihua Huang
|
6ead04a758
|
Merge pull request #524 from code4craft/proxyRefactor
HttpClient部分重构
|
2017-04-08 23:19:08 +08:00 |
|
yihua.huang
|
0f4d6e8b12
|
#525 remove port in UrlUtils.getDomain()
|
2017-04-08 23:17:00 +08:00 |
|
yihua.huang
|
a1ae632b62
|
test for request cookies and headers
|
2017-04-08 23:13:16 +08:00 |
|
yihua.huang
|
db67db8103
|
#523 remove fixAllRelativeHrefs by default, get absolute urls for links()
|
2017-04-08 22:06:18 +08:00 |
|
yihua.huang
|
abd020b45b
|
some comments
|
2017-04-08 20:16:17 +08:00 |
|
yihua.huang
|
2622b448b8
|
fix test
|
2017-04-08 20:09:43 +08:00 |
|
yihua.huang
|
b06a248c00
|
fix test
|
2017-04-08 20:06:04 +08:00 |
|
yihua.huang
|
1cfbd13aae
|
refacor in httpclientdownloader
|
2017-04-08 20:04:56 +08:00 |