[DUG] Work Wanted in Wellington

[Leigh Wanstead]

Hi Phil,

Thanks for the suggestion. Some of them are really nice. I never think of
it.

Yes, the scraper follow links.

Regards
Leigh


On 3 July 2014 15:45, Phil Scadden <p.scadden at gns.cri.nz> wrote:

> On 3/07/2014 2:01 p.m., Leigh Wanstead wrote:
> > I have a question for you guys related to how to be a good developer
> > to solve the problem :-)
> Well I have done website scrapping - no browser involved - but with the
> site owners blessing. Tough problem. Does the scraper follow links in
> your site? You could put in a link that is hidden by css and return
> garbage to anything that follows it. Make a very close study of header
> etc of anything that might give them away. Dont use it block - use it to
> generate garbage content for them instead.
>
> You could use js to get and display text with lazy loading. Or put text
> in image/flash. That makes it a pain in the neck for scrapers. Vary the
> dom layout at random but use matching css to make the appearance the
> same for  users - this doesnt stop scraping but makes finding the
> relevant information a pain because to the scraper, the website is
> constantly changing format.
>
>
> Notice: This email and any attachments are confidential.
> If received in error please destroy and immediately notify us.
> Do not copy or disclose the contents.
>
> _______________________________________________
> NZ Borland Developers Group - Delphi mailing list
> Post: delphi at listserver.123.net.nz
> Admin: http://delphi.org.nz/mailman/listinfo/delphi
> Unsubscribe: send an email to delphi-request at listserver.123.net.nz with
> Subject: unsubscribe
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://listserver.123.net.nz/pipermail/delphi/attachments/20140703/148c9e86/attachment-0001.html