Mobai
/
magic-Dependency
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,260 Commits
1 Branch
0 Tags
14 MiB
Commit Graph

597 Commits (5196a56ccf7aee374b44a02a1e9a414496431938)

Author SHA1 Message Date
Joe Zhou 5196a56ccf Format code. 2024-04-21 23:30:43 +08:00
Joe Zhou dba1668306 Add tests to test the equals & hashCode of Proxy. 2024-04-21 23:23:32 +08:00
Joe Zhou b7e0d360ec Upgrade junit from 4.13.2 to 5.10.2. 2024-04-21 22:04:48 +08:00
Sutra Zhou ed7429c293 Rename webmagic-parent to webmagic. 2024-04-03 19:37:56 +08:00
Sutra Zhou 4ebf48f6e3 Replace log4j 1.x with log4j 2.x, refs #534. 2024-04-03 18:26:01 +08:00
Joe Zhou 0ceaf14882 Bump version number from 0.10.1-SNAPSHOT to 1.0.0-SNAPSHOT for Java version updating from 1.8 to 11, refs #1134. 2024-04-03 01:00:47 +08:00
Sutra Zhou 31548deb93
Revert "Refactored code for increased optimization. (#1139)" (#1153) 
This reverts commit f051d978e2.
 2024-03-30 14:37:55 +08:00
Parthgajera056 f051d978e2
Refactored code for increased optimization. (#1139) 
* refactoring by decompose conditional technique

* refactoring by introduction explaining variable technique

* refactoring by rename method/variable technique

* refactoring by introducing explaining variable technique

* Added Extract class refactoring to increase maintainablilty

* Refactoring using replace conditional with polymorphism
 2024-03-30 14:28:02 +08:00
ayushi250317 9b9f173c1c
Refactored Code to increase maintainability (#1152) 
* Initial Commit

* Assignment 1 Submission

* Resolving Implementation Smells

* Refactoring Code to increase maintainability
 2024-03-30 14:26:41 +08:00
ayushi250317 28ac8bf9c4
Refactored Code to Resolve Implementation Code Smells (#1151) 
* Initial Commit

* Assignment 1 Submission

* Resolving Implementation Smells
 2024-03-29 00:45:12 +08:00
ayushi250317 80842d72db
Added test cases for improving line and branch coverage (#1150) 
* Initial Commit

* Assignment 1 Submission
 2024-03-23 13:22:30 +08:00
Sutra Zhou 95d1f44150 Optimize Request#extras, fix #1148. 2024-03-03 18:33:00 +08:00
Sutra Zhou 7ededbea1a Fix javadoc. 2023-12-05 12:56:14 +08:00
Sutra Zhou 1e5c6488ff Update for next development version 2023-12-05 12:48:14 +08:00
Sutra Zhou 73dd2ebbac Update versions for release 2023-12-05 12:28:05 +08:00
Harikrishna 7c20290ce4
Refactor addTargetRequests method to eliminate redundant code. (#1138) 2023-11-26 10:56:06 +08:00
Harikrishna a911104076
Refactored to remove multiple calls of getSourceTexts() api (#1137) 2023-11-24 20:09:32 +08:00
Harikrishna 622ed5a17f
Refactor compareLong method using Long.compare, corrected the local variable name (#1136) 2023-11-24 12:37:04 +08:00
Sutra Zhou 67644de3d9 Expose Page to onSuccess & onError. 2023-11-20 18:28:27 +08:00
Joe Zhou eda3be9432 Fix log format. 2023-11-06 22:44:53 +08:00
Joe Zhou 8c008563ff Update for next development version 2023-09-10 17:06:55 +08:00
Joe Zhou 07e83a421b Update versions for release 2023-09-10 16:35:32 +08:00
Joe Zhou 6e8d130113 Update for next development version 2023-06-22 11:23:44 +08:00
Joe Zhou 3688226e32 Update versions for release 2023-06-22 11:16:41 +08:00
Joe Zhou ad010927f6 Fix test. 2023-06-22 10:40:46 +08:00
Joe Zhou 58fd08bcf8 Expose Request to ProxyProvider. 2023-06-22 09:52:58 +08:00
Tanky-Zhang 244ade7b4c
feat:update host verify (#1112) 2023-03-22 22:25:51 +08:00
hooyantsing 12ce86425f BugFix: Jsoup 和 HtmlCleaner 构建 Dom 时，若缺失 table 标签，则无法正常解析 tr 和 td 标签。 2023-02-03 22:48:58 +08:00
Joe Zhou a266df406f Add Site.defaultCharset. closes #1101. 2022-12-20 23:41:31 +08:00
Joe Zhou 80424b0bd7 Replace List with Iterable, fixed #1099. 2022-12-05 23:26:01 +08:00
Joe Zhou cda8467f95 Update for next development version 2022-11-24 00:49:43 +08:00
Joe Zhou 888682863c Update versions for release 2022-11-24 00:30:07 +08:00
Joe Zhou 64e6a9800a Add dead-lock note for QueueScheduler. 2022-11-24 00:26:09 +08:00
Joe Zhou e735e4e585 Log the remaining capacity. 2022-11-20 18:31:36 +08:00
Joe Zhou 4915431845 Revise logging level from warn to info, as we have passed the exception to onError. refs #1094 2022-11-15 22:48:02 +08:00
Joe Zhou 075b98291b Return spider in setEmptySleepTime itself for chainning. 2022-11-12 11:06:08 +08:00
Joe Zhou ac912e8f1f Revise QueueScheduler to support capacity-restricted. 2022-11-12 10:17:36 +08:00
Sutra Zhou 5d5f3bf20e Update for next development version 2022-10-24 00:17:21 +08:00
Sutra Zhou 838c47f1f6 Update versions for release 2022-10-23 23:58:21 +08:00
Sutra Zhou d2b2eed9df Pass the task to onSuccess & onError. 2022-10-19 22:15:41 +08:00
Sutra Zhou 5f80e02abd Interrupt current thread. 2022-10-19 22:08:38 +08:00
Sutra Zhou 126c32ecd0 Fix compatible issue. 2022-10-16 07:21:35 +08:00
vio.ao e7a7fbeeeb Enhance Jsoup could parse tr td tag directly 2022-10-01 18:23:15 +08:00
vio.ao 7a62a6cb45 Revert "Revert "Common the downloader status process and pass error information when …"" 
This reverts commit acfbd7b883.
 2022-10-01 17:33:11 +08:00
Sutra Zhou acfbd7b883
Revert "Common the downloader status process and pass error information when …" 2022-10-01 10:37:09 +08:00
vio.ao d01f26333b Common the downloader status process and pass error information when onError 2022-10-01 00:21:17 +08:00
David Hsing 54da7af17e change dependency versions into properties 
change dependency versions into properties
update commons-collections from 3.x to 4.4
 2022-05-03 17:42:42 +08:00
Sutra Zhou 692605bd75 Merge branch 'develop' of github.com:code4craft/webmagic into develop 2021-10-24 23:20:56 +08:00
Sutra Zhou 34da2fb3a0 Make PageProcessor#getSite be default method. Closes #1040. 2021-10-24 23:20:38 +08:00
carl.don:tjr fcdb9074d6 格式化 Spider.run 代码 2021-08-04 18:23:04 +08:00