☻ 唐鳳 ☺ > YiJing > YiJing::0x11

Download:
YiJing-0.10.tar.gz

Annotate this POD

View/Report Bugs
Source  

NAME ^

YiJing::0x11 - Gu\/ (Web Crawler)

DESCRIPTION ^

"Beware: There is only a thin line between a crawler and a worm!"

Judgment

    蠱。元。亨。利涉大川。先甲三日。后甲三日。

Web crawler: Nice and fun. Suitable to sift in the great Data Flow. Test run for three days before sending it out; analyze the data for three days before sending it out again.

Interpretation

    彖曰。蠱。剛上而柔下。巽而止。
          蠱。元亨。而天下治也。利涉大川。往有事也。
          先甲三日。后甲三日。終則有始。天行也。

This hexagram is emblematic of the trouble that you would face in writing or managing a web crawler: the program has to strike out on its own and unobtrusively sift great gobs of data in any number of messy formats.

It needs testing and retesting, planning and monitoring. It has to follow old standards and accept new ones, and tolerate sites that don't follow standards at all. To do its work, the web crawler has to get as much data from as many sites as it can, without bothering any webmasters in the process.

It has to be efficient, but deliberate. It is a matter of contradictory goals -- a situation that comes up in all sorts of systems besides web crawlers.

Image

    象曰。山下有風。蠱。君子以振民育德。

Gathering Data under Standards, is the Image of a Web Crawler. A wise hacker makes careful use of it to provide people with interesting information while maintaining the proper ethics.

Lines

syntax highlighting: