yihua.huang
|
6c11718566
|
Clean project structure #70
|
2014-03-14 23:24:38 +08:00 |
|
yihua.huang
|
0e98183f74
|
Change log4j to slf4j #55
|
2014-02-12 09:35:57 +08:00 |
|
yihua.huang
|
fa33b15843
|
property loader
|
2014-02-11 23:07:31 +08:00 |
|
yihua.huang
|
362fdd0662
|
Merge branch 'master' of github.com:code4craft/webmagic
|
2014-02-11 22:23:56 +08:00 |
|
yihua.huang
|
af809c4d55
|
update version to 0.5.0-snapshot
|
2014-02-11 22:16:01 +08:00 |
jon
|
a722f9bb66
|
修复由于FileCacheQueueScheduler中fileCursor 文件再次打开时没有初始化抛出NullPointerException的错误
|
2014-01-08 21:24:58 +08:00 |
|
yihua.huang
|
486d9d276f
|
#45 Remove multi in ExtractBy
|
2013-11-28 18:23:51 +08:00 |
|
yihua.huang
|
18a3af4a0a
|
add more sample for jsonpath #42
|
2013-11-28 09:58:22 +08:00 |
|
yihua.huang
|
59ad4cad27
|
#42 Add jsonpath in annotation mode for json result
|
2013-11-28 08:25:16 +08:00 |
|
yihua.huang
|
cf62d707e0
|
#36 Spider does not exit when success
|
2013-11-27 23:33:18 +08:00 |
|
yihua.huang
|
a01312930a
|
#39 Parsing html after page.getHtml()
|
2013-11-27 22:01:34 +08:00 |
|
yihua.huang
|
b838c4e433
|
#34 Close reader in FileCacheQueueScheduler
|
2013-11-08 14:59:09 +08:00 |
|
yihua.huang
|
fd6d2fd6f8
|
try to keepalive TCP connection
|
2013-11-06 21:19:14 +08:00 |
|
yihua.huang
|
e046bb0723
|
remove useless code
|
2013-11-06 12:48:14 +08:00 |
|
yihua.huang
|
6e32a19f80
|
update api for direct download
|
2013-11-06 12:46:50 +08:00 |
|
yihua.huang
|
807aefe9df
|
change EntityUtil to IOUtil because some encoding error
|
2013-11-06 07:37:34 +08:00 |
|
yihua.huang
|
8f774afc84
|
add direct download
|
2013-11-06 06:41:04 +08:00 |
|
yihua.huang
|
2e496402dc
|
add more warning for 0.3.3
|
2013-10-24 13:16:48 +08:00 |
|
yihua.huang
|
1a2c84ea78
|
#27 add timeout config to site
|
2013-10-11 07:36:16 +08:00 |
|
yihua.huang
|
3b00190f99
|
api without implementation for #28: add specific url crawl
|
2013-10-10 00:40:44 +08:00 |
|
yihua.huang
|
6f18eec77e
|
fix a test error
|
2013-09-23 13:07:33 +08:00 |
|
yihua.huang
|
b131878123
|
add example
|
2013-09-23 13:01:28 +08:00 |
|
yihua.huang
|
95ab4edec3
|
some bugfix
|
2013-09-23 08:38:54 +08:00 |
|
yihua.huang
|
250cc5e662
|
change formatter to class
|
2013-09-23 08:17:21 +08:00 |
|
yihua.huang
|
b18216245b
|
add type convert
|
2013-09-23 07:53:33 +08:00 |
|
yihua.huang
|
d7c7a78177
|
complete test cases
|
2013-09-08 22:19:02 +08:00 |
|
yihua.huang
|
c17a31a21d
|
fix null pointe exception #26
|
2013-09-08 21:09:49 +08:00 |
|
yihua.huang
|
d141541ef3
|
add retry
|
2013-09-04 09:57:19 +08:00 |
|
yihua.huang
|
aefd0569a5
|
update version
|
2013-09-04 09:36:56 +08:00 |
|
yihua.huang
|
194518fd82
|
add switch
|
2013-09-04 08:21:34 +08:00 |
|
yihua.huang
|
326b97c65a
|
update
|
2013-09-04 00:15:54 +08:00 |
|
yihua.huang
|
d7cd9e5747
|
update pom
|
2013-09-02 11:56:01 +08:00 |
|
yihua.huang
|
478ace7e97
|
add FilePageModelPipeline
|
2013-08-22 07:29:18 +08:00 |
|
yihua.huang
|
09ffd468c0
|
fix comments
|
2013-08-20 22:53:16 +08:00 |
|
yihua.huang
|
c70ed57025
|
remove PriorityScheduler to core
|
2013-08-20 21:55:58 +08:00 |
|
yihua.huang
|
7003426898
|
update pom
|
2013-08-20 21:52:39 +08:00 |
|
yihua.huang
|
c79d6ecf09
|
complete all comments
|
2013-08-17 23:30:49 +08:00 |
|
yihua.huang
|
5073258237
|
closable
|
2013-08-17 21:19:24 +08:00 |
|
yihua.huang
|
5f1f4cbc46
|
update comments
|
2013-08-17 20:41:29 +08:00 |
|
yihua.huang
|
6cc1d62a08
|
bugfix: rawhtml do not work
|
2013-08-17 19:42:51 +08:00 |
|
yihua.huang
|
a994b1c9fd
|
complete extension comments in en
|
2013-08-17 19:35:45 +08:00 |
|
yihua.huang
|
c59c1fe80d
|
update comments
|
2013-08-17 19:19:27 +08:00 |
|
yihua.huang
|
59aad6a7f4
|
comments in english
|
2013-08-17 18:33:05 +08:00 |
|
yihua.huang
|
e566a53936
|
update ignore test
|
2013-08-17 18:13:13 +08:00 |
|
yihua.huang
|
1148450ff9
|
update filecache to more useful
|
2013-08-17 18:12:47 +08:00 |
|
yihua.huang
|
3ba7a76f44
|
add combo extract to replace Extract2 Extract3...
|
2013-08-17 17:23:11 +08:00 |
|
yihua.huang
|
5cb45af3a4
|
+doc
|
2013-08-17 12:10:34 +08:00 |
|
yihua.huang
|
a339e4ab5c
|
add jsonpathselector
|
2013-08-12 13:36:44 +08:00 |
|
yihua.huang
|
9e82256ce3
|
update docs
|
2013-08-12 10:08:20 +08:00 |
|
yihua.huang
|
f21097421b
|
add new constructor to redisscheduler
|
2013-08-11 18:53:13 +08:00 |