[DUG] Work Wanted in Wellington

Fri Jul 4 11:12:16 NZST 2014

HI Leigh

Could you share the website address with us.  I think if we could see the address and put your problem into context we would be able to offer solutions that better fit your requirements.

Cameron Hart

Flow Software Limited

[Flow]

PO Box 302 768, North Harbour

P

+64 9 476 3569

Auckland 0751, New Zealand

M

+64 21 222 3569

www.flowsoftware.co.nz <http://www.flowsoftware.co.nz>

E

cameron.hart at flowsoftware.co.nz <mailto:cameron.hart at flowsoftware.co.nz>

This message is intended for the addressee named above. It may contain privileged or confidential information. If you are not the intended recipient of this message you must not use, copy, distribute or disclose it to anyone.

  P Please consider the environment before printing this email

From: delphi-bounces at listserver.123.net.nz [mailto:delphi-bounces at listserver.123.net.nz] On Behalf Of Leigh Wanstead
Sent: Friday, 4 July 2014 10:57 a.m.
To: NZ Borland Developers Group - Delphi List
Subject: Re: [DUG] Work Wanted in Wellington

Hi John,

Thanks for the suggestion.

But if I offer authenticate after 100-200 records, the hacker just give up that cookie, ip and get a new ip to bypass the restriction.

Regards
Leigh

On 4 July 2014 10:51, John Bird <johnkbird at paradise.net.nz<mailto:johnkbird at paradise.net.nz>> wrote:
Sounds to me like the data needs to be hidden from the web site in a database with authentication to view records.   You could easily allow users to see say some eg 100-200 records even if not authenticated but not more.

There is a similar restriction in data for instance with electoral roles – you can look up names but not scrape data – there the reasons are for privacy as well as to protect their income as they sell data to certain users only.

From: Leigh Wanstead<mailto:leigh.wanstead at gmail.com>
Sent: Friday, July 4, 2014 10:39 AM
To: NZ Borland Developers Group - Delphi List<mailto:delphi at listserver.123.net.nz>
Subject: Re: [DUG] Work Wanted in Wellington

Hi David,

It is like amazon. Amazon does not require user name/password just browsing the data.

Regards
Leigh

On 4 July 2014 10:29, David Brennan <dugdavid at dbsolutions.co.nz<mailto:dugdavid at dbsolutions.co.nz>> wrote:
Sounds unusual. So the company sells the data but doesn’t have a login system to control who consumes the data?

David.

From: delphi-bounces at listserver.123.net.nz<mailto:delphi-bounces at listserver.123.net.nz> [mailto:delphi-bounces at listserver.123.net.nz<mailto:delphi-bounces at listserver.123.net.nz>] On Behalf Of Leigh Wanstead
Sent: Friday, 4 July 2014 10:16 a.m.

To: NZ Borland Developers Group - Delphi List
Subject: Re: [DUG] Work Wanted in Wellington

Hi Jolyon,

The company I work for is selling data. The data is the income of the company.

Regards
Leigh

On 4 July 2014 09:23, Jolyon Smith <jsmith at deltics.co.nz<mailto:jsmith at deltics.co.nz>> wrote:
I don't understand this determination to make the hacker's life difficult.  Surely the objective is to address the impact on the site for legitimate users ?

Contriving schemes to make the hacker's life difficult is simply extending the problem domain into an irrelevant area and increasing the complexity by orders of magnitude in order to protect information that is public already - there is no mention of any attempt to thwart site security, only scraping of publicly accessible URL's..

If the intent is to disincentivise the hacker, simply denying them the ability to scrape the site by detecting and blocking them will cause them inconvenience enough.  Even if it doesn't, as long as their activity is not impacting on the legitimate operation of the site then the key objective is met - that of maintaining site response for legit users.

Almost all of these schemes to make the scrapers life miserable do also impact on the legitimate user experience, loading up the server and the client browser processing with overhead targeted at the scraper but imposed on ALL clients.

I can see that the technical challenge of "beating" the hacker could be attractive, but it seems to me to be an ultimately pointless and resource sapping "Arms Race" that cannot ever really be won...  even if you eventually drive the scraper to give up entirely, burdensome counter-measures will themselves have impacted on your site, defeating if not the whole object then certainly a significant part of it, of getting rid of the scraper activity in the first place.

Of course, if you can find counter-measures which do not impose any such burden on legit users then you have the best of both worlds, but the key need to be met is addressing the scraper by removing the impact on legit users, not adding to it.

So, bringing it back to the original topic - What makes a good developer ?

Another characteristic would be the ability to remain focused on the key objective/user need, rather than being drawn into a bottomless honey pot of technical challenge of limited/no direct relevance to the problem at hand.

:)

On 4 July 2014 09:05, Phil Scadden <p.scadden at gns.cri.nz<mailto:p.scadden at gns.cri.nz>> wrote:

> Regarding to render the website in Javascript, how are you going to
> stop the browser driven by script? The hacker does not need to
> understand the javascript. All he need is just grab dom element.
That would be true but very unlikely that hacker is using browser. Too
slow. If you load the html with junk data and modify it with js, it may
take the hacker a long time to notice they are using crap. But I would
looking at detecting the hacker without a tip off in first place and
then figure out ways to make life difficult.

--
Phil Scadden, Senior Scientist GNS Science Ltd 764 Cumberland St,
Private Bag 1930, Dunedin, New Zealand Ph +64 3 4799663<tel:%2B64%203%204799663>, fax +64 3 477 5232<tel:%2B64%203%20477%205232>

Notice: This email and any attachments are confidential.
If received in error please destroy and immediately notify us.
Do not copy or disclose the contents.

_______________________________________________
NZ Borland Developers Group - Delphi mailing list
Post: delphi at listserver.123.net.nz<mailto:delphi at listserver.123.net.nz>
Admin: http://delphi.org.nz/mailman/listinfo/delphi
Unsubscribe: send an email to delphi-request at listserver.123.net.nz<mailto:delphi-request at listserver.123.net.nz> with Subject: unsubscribe

_______________________________________________
NZ Borland Developers Group - Delphi mailing list
Post: delphi at listserver.123.net.nz<mailto:delphi at listserver.123.net.nz>
Admin: http://delphi.org.nz/mailman/listinfo/delphi
Unsubscribe: send an email to delphi-request at listserver.123.net.nz<mailto:delphi-request at listserver.123.net.nz> with Subject: unsubscribe

_______________________________________________
NZ Borland Developers Group - Delphi mailing list
Post: delphi at listserver.123.net.nz<mailto:delphi at listserver.123.net.nz>
Admin: http://delphi.org.nz/mailman/listinfo/delphi
Unsubscribe: send an email to delphi-request at listserver.123.net.nz<mailto:delphi-request at listserver.123.net.nz> with Subject: unsubscribe

________________________________
_______________________________________________
NZ Borland Developers Group - Delphi mailing list
Post: delphi at listserver.123.net.nz<mailto:delphi at listserver.123.net.nz>
Admin: http://delphi.org.nz/mailman/listinfo/delphi
Unsubscribe: send an email to delphi-request at listserver.123.net.nz<mailto:delphi-request at listserver.123.net.nz> with Subject: unsubscribe

_______________________________________________
NZ Borland Developers Group - Delphi mailing list
Post: delphi at listserver.123.net.nz<mailto:delphi at listserver.123.net.nz>
Admin: http://delphi.org.nz/mailman/listinfo/delphi
Unsubscribe: send an email to delphi-request at listserver.123.net.nz<mailto:delphi-request at listserver.123.net.nz> with Subject: unsubscribe

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://listserver.123.net.nz/pipermail/delphi/attachments/20140703/2a1b6b2b/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 648 bytes
Desc: image001.jpg
Url : http://listserver.123.net.nz/pipermail/delphi/attachments/20140703/2a1b6b2b/attachment-0002.jpg 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 4211 bytes
Desc: image002.jpg
Url : http://listserver.123.net.nz/pipermail/delphi/attachments/20140703/2a1b6b2b/attachment-0003.jpg 