Commit Graph

630 Commits (41c2ea94984ed19120d9ef0abe2ef1d0b93135fd)

Author SHA1 Message Date
yihua.huang 04fde8203b add control for monitor 2014-04-26 11:44:14 +08:00
yihua.huang 2770811a10 update monitor example 2014-04-26 11:24:22 +08:00
yihua.huang 5ecd909ef2 add timeout for wait/notify #111 2014-04-25 19:41:36 +08:00
yihua.huang 964e637264 fix ut #110 2014-04-25 18:46:31 +08:00
yihua.huang c7afdb516e remove thread utils #110 2014-04-25 18:44:45 +08:00
yihua.huang 17e95f2a7f comments 2014-04-25 18:39:01 +08:00
yihua.huang 3a666fcebf add sample of 36kr #110 2014-04-25 18:36:07 +08:00
yihua.huang b1258f4f16 remove some test 2014-04-25 18:30:49 +08:00
yihua.huang 05eb7831b6 refactor and comments #110 2014-04-25 18:27:40 +08:00
yihua.huang 375e64e845 more monitor status 2014-04-25 18:10:14 +08:00
yihua.huang 018061d2cd fix error in thread pool 2014-04-25 18:01:02 +08:00
yihua.huang cdc423f2bf log 2014-04-25 17:41:41 +08:00
yihua.huang c6661899fd new thread pool #110 2014-04-25 17:33:48 +08:00
yihua.huang 179baa7a22 return when page is null 2014-04-25 16:07:41 +08:00
yihua.huang 0336f4cdb4 remove IllegalStateException when download error for less error log 2014-04-25 16:06:29 +08:00
yihua.huang 4738ae2d14 change url find to match #94 2014-04-25 16:04:41 +08:00
yihua.huang f973889cda refactor subpageprossor etc. #94 2014-04-25 15:48:05 +08:00
yihua.huang acb63d55d7 some check and example #98 2014-04-25 13:26:08 +08:00
yihua.huang 11ba5beb42 [refactor]move monitor to webmagic-extension #98 2014-04-25 13:17:13 +08:00
yihua.huang d61f65cef8 update mbean to mxbean #98 2014-04-25 11:31:43 +08:00
yihua.huang ad6a273b12 update test url 2014-04-25 11:28:35 +08:00
yihua.huang 30af23d003 split monitor to server and client mode #98 2014-04-25 11:25:52 +08:00
yihua.huang ced79630d3 specify jndi and jmx #98 2014-04-25 11:11:15 +08:00
yihua.huang 95d3802e77 add formdata support for post request #108 2014-04-24 11:48:58 +08:00
yihua.huang f49bb877c8 clean some code #109 2014-04-24 11:38:13 +08:00
yihua.huang e1aaf1dd11 fix mistake of guava Table #109 2014-04-24 11:05:49 +08:00
yihua.huang 8ba2da146c request method #108 and more cookie #109 config 2014-04-24 10:51:37 +08:00
yihua.huang b06aa489fb [BugFix]Only one url from sourceRegion can be extracted #107 2014-04-18 17:48:26 +08:00
Bo LIANG 08fa3b01c1 when download error, throw an exception instead of calling onError and returning peacefully. #105 2014-04-17 17:53:12 +08:00
yihua.huang 023c2ac84e spider config draft 2014-04-17 16:44:32 +08:00
yihua.huang 27b37e8164 extension point and sample for JMX support #98 2014-04-17 08:12:37 +08:00
yihua.huang a5db6cf292 some monitor and JMX support #98 2014-04-17 00:35:09 +08:00
yihua.huang f39aa435cf add null check #104 2014-04-16 19:46:32 +08:00
yihua.huang 42bbe40a37 [Bugfix]Urls will be lost when call setScheduler() #104 2014-04-16 19:45:17 +08:00
yihua.huang aae1ab2cd6 fix compile error 2014-04-16 18:14:13 +08:00
yihua.huang bc8d0220eb Merge branch 'master' of github.com:code4craft/webmagic 2014-04-16 18:13:51 +08:00
yihua.huang 1fbfc92de2 Inherit support of Field annotation in Model #103 2014-04-16 18:13:44 +08:00
Yihua Huang 93c4a2afb7 Merge pull request #102 from ccliangbo/waitNewUrl
combine two try-catch block into one, make it cleaner.
2014-04-16 16:38:10 +08:00
Bo LIANG 163773af6b combine two try-catch block into one, make it cleaner. 2014-04-16 16:05:08 +08:00
yihua.huang c8014a9ae6 update readme 2014-04-15 15:34:37 +08:00
yihua.huang ec446277b1 some refactor in httpclientdownloader 2014-04-15 15:30:37 +08:00
yihua.huang 4a035e729a extension point for LocalDuplicatedRemovedScheduler #95 2014-04-13 23:31:13 +08:00
yihua.huang b249e49748 [Bugfix]loop error when add TargetRequest #99 2014-04-13 23:04:09 +08:00
yihua.huang 3a79b1b64a [Bugfix]formatter property does not work when field is String#100 2014-04-13 23:02:34 +08:00
Yihua Huang cc9d319fd9 Merge pull request #94 from sebastian1118/master
update:PatternHandler
2014-04-13 13:16:20 +08:00
Yihua Huang da2f023c12 Merge pull request #96 from ouyanghuangzheng/master
修改了Spider 和site  几处注释
2014-04-13 13:12:12 +08:00
yihua.huang f7950ebcab fix tests 2014-04-13 13:00:31 +08:00
yihua.huang b14f0ee479 fix jsonpath in AngularJSProcessor 2014-04-13 12:54:44 +08:00
愤怒的番茄 32ba1b8889 修复几处注释问题 2014-04-13 12:41:15 +08:00
yihua.huang 84b897f83b update AngularJSProcessor 2014-04-13 12:20:57 +08:00