3a79b1b64a
[Bugfix]formatter property does not work when field is String#100
2014-04-13 23:02:34 +08:00
cc9d319fd9
Merge pull request #94 from sebastian1118/master
...
update:PatternHandler
2014-04-13 13:16:20 +08:00
da2f023c12
Merge pull request #96 from ouyanghuangzheng/master
...
修改了Spider 和site 几处注释
2014-04-13 13:12:12 +08:00
f7950ebcab
fix tests
2014-04-13 13:00:31 +08:00
b14f0ee479
fix jsonpath in AngularJSProcessor
2014-04-13 12:54:44 +08:00
32ba1b8889
修复几处注释问题
2014-04-13 12:41:15 +08:00
84b897f83b
update AngularJSProcessor
2014-04-13 12:20:57 +08:00
03c251237b
add Json parse support
2014-04-13 10:23:00 +08:00
99e12aafaa
update:PatternHandler
2014-04-13 10:14:39 +08:00
53184f0390
test
2014-04-12 23:00:37 +08:00
644e8d1f72
同步官方源码
2014-04-12 22:32:22 +08:00
610ac42c07
更新
2014-04-12 22:22:07 +08:00
5b254e446b
更新
2014-04-12 22:08:53 +08:00
843e928c2c
comments on sinablogprocessor sample
2014-04-12 20:10:24 +08:00
be37d8b216
sinablogprocessor sample
2014-04-12 20:03:44 +08:00
094f9d1552
rename assets for spell mistake
2014-04-12 13:42:32 +08:00
2b023c95c2
qqmeishi demo
2014-04-11 11:43:04 +08:00
db65dfafb8
add baidunews sample
2014-04-09 23:32:07 +08:00
3669e73e4a
update News163: use Xsoup 0.2.0 syntax instead of ComboExtract
2014-04-09 16:43:55 +08:00
02b441ad38
disable NativeObject in Rhino because it is a hotspot internal api and compile error in OpenJDK #93
2014-04-09 15:40:33 +08:00
9f5a6494a0
add support for JDK6 #93
2014-04-09 10:44:52 +08:00
c6c56ad511
Merge branch 'master' of github.com:code4craft/webmagic
2014-04-09 09:54:13 +08:00
c2873928c8
[prototype] extractrule
2014-04-09 09:54:01 +08:00
7cb4e37812
Merge pull request #93 from friddle/master
...
update the script
2014-04-07 23:22:35 +08:00
933800147b
update ruby
2014-04-07 23:18:00 +08:00
37666a7151
update the script
2014-04-07 23:04:24 +08:00
c1e7207869
add FileCacheQueueScheduler support for cycleRetryTimes
2014-04-07 11:00:09 +08:00
969ad1766b
change logger style to slf4j for cleaner code
2014-04-06 21:32:20 +08:00
9b2cb43f47
ConfigurablePageProcessor #91
2014-04-05 23:40:10 +08:00
1090d070d9
Merge pull request #90 from ccliangbo/removeUnusedLines
...
Remove unused variable to make the project cleaner.
2014-04-05 22:00:30 +08:00
159eeea2f5
Remove unused variable to make the project cleaner.
2014-04-05 18:32:12 +08:00
c143fc662c
add SubPageProcessor #86
2014-04-05 18:17:48 +08:00
2b2ce9ce13
Merge pull request #89 from ccliangbo/slf4jFormat
...
change the formatter of log.
2014-04-05 15:11:58 +08:00
b043ac76d6
change the formatter of log.
...
To use slf4j, we should insert {} into the formatter string.
2014-04-05 11:31:56 +08:00
474f785dab
Merge pull request #86 from sebastian1118/master
...
new feature: PatternProcessor
2014-04-04 23:41:27 +08:00
8fe967ba8d
[BugFix]exclude log4j.xml from maven jar plugin #82
2014-04-04 23:39:32 +08:00
38a12f8641
new feature: PatternProcessor
2014-04-04 22:02:52 +08:00
dafd0b5875
[BugFix]multi model in one pageprocessor will be skipped #85
2014-04-04 20:36:31 +08:00
7aaf837e15
change logger to slf4j style for performance #84
2014-04-04 20:10:00 +08:00
f9b157951d
Merge branch 'master' of github.com:code4craft/webmagic
2014-04-04 20:01:14 +08:00
22c394e629
[doc]
2014-04-04 20:00:58 +08:00
3efa774191
Merge pull request #84 from ccliangbo/logInScheduler
...
Modify the log levels of LocalDuplicatedRemovedScheduler.java
2014-04-04 17:16:34 +08:00
762a3973fd
Modify the log levels of LocalDuplicatedRemovedScheduler.java
...
The old version will print a debug level log each time the push method is
called. So sometimes, when a html page have multiple links for the same
page, the debug log will appears more than once. Also, when we meet a
duplicate URL, it will also print a log, which will be confusing.
I change the level of it to trace. And each time a URL is really push into
queue, print a debug level log.
2014-04-04 15:53:46 +08:00
44293cd894
[doc]add qq group in readme
2014-04-04 10:07:48 +08:00
9a0a4051ed
[doc] ch3 part1
2014-04-04 08:05:34 +08:00
7ca644cdd9
format readme
2014-04-04 06:47:28 +08:00
a1c7e826f7
fix dep of slf4j-log4j12
2014-04-03 23:04:31 +08:00
a34e92d11a
fix huabanprocessor
2014-04-03 22:33:10 +08:00
50cee4c7bb
[doc] complete docs2.0 ch1
2014-04-03 11:06:03 +08:00
4abdff5dd4
add eclipse project
2014-04-03 09:56:02 +08:00
yihua.huang
愤怒的番茄
Tian
friddle
Bo LIANG