Mobai
/
magic-Dependency
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #84 from ccliangbo/logInScheduler 

Modify the log levels of LocalDuplicatedRemovedScheduler.java
master
Yihua Huang 2014-04-04 17:16:34 +08:00
parent 44293cd894 762a3973fd
commit 3efa774191
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
webmagic-core/src/main/java/us/codecraft/webmagic/scheduler/LocalDuplicatedRemovedScheduler.java
View File

@ -23,8 +23,9 @@ public abstract class LocalDuplicatedRemovedScheduler implements Scheduler {
@Override
public void push(Request request, Task task) {
logger.debug("push to queue " + request.getUrl());
logger.trace("get a candidate url " + request.getUrl());
if (request.getExtra(Request.CYCLE_TRIED_TIMES) != null || urls.add(request.getUrl())) {
logger.debug("push to queue " + request.getUrl());
pushWhenNoDuplicate(request, task);
}
}