Commit Graph

773 Commits (b9544424f240d127ee71ce088caa8d5332b67749)

Author SHA1 Message Date
yihua.huang b33f84fdc9 add distributed architechure 2014-03-12 22:50:03 +08:00
yihua.huang 55368919df add attribute 'text' support for CssSelector #66 2014-03-11 13:18:34 +08:00
yihua.huang 88b50d4182 bigfix: cycleTry will not work when spawnUrl is set to false #62 2014-03-04 07:33:07 +08:00
yihua.huang 16ae066c62 update sample code in readme #63 2014-03-03 21:22:39 +08:00
yihua.huang 4d628d80b4 add redis scheduler 2014-03-03 08:24:55 +08:00
yihua.huang ca29cd3ef2 add jokejimodel for example 2014-03-03 08:22:14 +08:00
yihua.huang f094f31816 update contributors 2014-03-01 15:31:13 +08:00
yihua.huang 2768a1cae4 add test for cycleTriedTimes and fix cycleTriedTimes inc error #60 2014-03-01 15:10:38 +08:00
yihua.huang bbd0d7e600 update httpclient version to 4.3.3 #59 2014-02-28 21:17:02 +08:00
yihua.huang 571061454a #58 add CYCLE_TRIED_TIMES support to QueueScheduler and PriorityScheduler 2014-02-27 23:54:30 +08:00
yihua.huang a79ae6a9e4 add ui design for avalon 2014-02-27 23:29:25 +08:00
yihua.huang 0b436aca4f remove fiel 2014-02-24 22:48:55 +08:00
yihua.huang b93cc92fb5 pages for spiders 2014-02-24 22:47:51 +08:00
yihua.huang cd0b3a4df4 invite ui from charisma 2014-02-23 23:12:58 +08:00
Yihua Huang 19843c063b Merge pull request #57 from d0ngw/master
Fix chrome driver can't quit
2014-02-17 07:17:45 +08:00
d0ngw 4fc89485b6 Fix chrome driver can't quit 2014-02-16 22:04:12 +08:00
yihua.huang 0e98183f74 Change log4j to slf4j #55 2014-02-12 09:35:57 +08:00
yihua.huang fa33b15843 property loader 2014-02-11 23:07:31 +08:00
yihua.huang 362fdd0662 Merge branch 'master' of github.com:code4craft/webmagic 2014-02-11 22:23:56 +08:00
yihua.huang af809c4d55 update version to 0.5.0-snapshot 2014-02-11 22:16:01 +08:00
Yihua Huang 7e8a5c7dd2 Merge pull request #54 from bitdeli-chef/master
Add a Bitdeli Badge to README
2014-01-21 23:00:07 -08:00
Bitdeli Chef 6cade5ddf3 Add a Bitdeli badge to README 2014-01-22 07:03:10 +00:00
Yihua Huang 31ff4cc404 Merge pull request #53 from xuchaoo/master
修复由于FileCacheQueueScheduler中fileCursor 文件再次打开时没有初始化抛出NullPointerExceptio...
2014-01-08 17:49:44 -08:00
jon a722f9bb66 修复由于FileCacheQueueScheduler中fileCursor 文件再次打开时没有初始化抛出NullPointerException的错误 2014-01-08 21:24:58 +08:00
Yihua Huang 090827f124 Merge pull request #52 from d0ngw/master
The SeleniumDownloader should call the setRawText
2013-12-26 19:30:31 -08:00
d0ngw a5a9b141b3 The SeleniumDownloader should call the setRawText 2013-12-27 11:09:04 +08:00
yihua.huang 6933029ea5 update modules 2013-12-25 18:48:31 +08:00
Yihua Huang 5d9fda0614 Merge pull request #51 from code4craft/test
Update RegexSelector.java
2013-12-20 16:43:49 -08:00
Almark Ming 2b46b11e55 Update RegexSelector.java
Optimize regex format check

Conflicts:
	webmagic-core/src/main/java/us/codecraft/webmagic/selector/RegexSelector.java
2013-12-21 08:38:17 +08:00
yihua.huang 2a8e1b654d Merge branch 'master' of git.oschina.net:flashsword20/webmagic into osc
Conflicts:
	pom.xml
2013-12-21 07:59:28 +08:00
yihua.huang d9a15ad66a 404 page 2013-12-21 07:56:58 +08:00
黄亿华 1bdffc1e56 Merge pull request !337 from Almark Ming/master 2013-12-21 07:54:57 +08:00
yihua.huang 12a6390cbd update spring4 configuration 2013-12-18 01:02:59 +08:00
yihua.huang 7fd27d9b22 add spring 4 mvc for worker 2013-12-17 23:57:23 +08:00
Almark Ming 91ed66ecac Update RegexSelector.java 2013-12-17 16:57:22 +08:00
Almark Ming 83926970b2 Check valid left parenthesis 2013-12-17 16:55:53 +08:00
yihua.huang 31fb0048a1 add worker 2013-12-07 00:37:07 +08:00
yihua.huang b51fb2696b update ut for cookie 2013-12-06 00:30:01 +08:00
yihua.huang ff2f588c41 #48 nullpointer exception 2013-12-04 22:11:20 +08:00
yihua.huang 0c3ff3d6b1 remove duplicate logo in readme 2013-12-04 00:05:38 +08:00
yihua.huang fc97cb58c5 update lib and version 2013-12-04 00:04:29 +08:00
yihua.huang 7c41bec92f Merge branch 'master' of github.com:code4craft/webmagic
Conflicts:
	README.md
	webmagic-samples/pom.xml
	webmagic-selenium/pom.xml
2013-12-03 23:50:26 +08:00
yihua.huang ac516f9b0e update version in docs 2013-12-03 23:46:31 +08:00
yihua.huang cc241bc0f2 update versions 2013-12-03 23:40:47 +08:00
yihua.huang d274310cb2 [maven-release-plugin] prepare for next development iteration 2013-12-03 23:35:06 +08:00
yihua.huang e8c32a32dc [maven-release-plugin] prepare release webmagic-0.4.2 2013-12-03 23:34:57 +08:00
yihua.huang 93cb4308ef update pom 2013-12-03 23:29:57 +08:00
yihua.huang 6a828e923c #46 Downloader thread hang up when timeout 2013-12-03 09:59:54 +08:00
yihua.huang 486d9d276f #45 Remove multi in ExtractBy 2013-11-28 18:23:51 +08:00
yihua.huang aaa53f58c7 Merge branch 'master' of github.com:code4craft/webmagic 2013-11-28 16:27:49 +08:00