<div dir="ltr">Hi John,<div><br></div><div>Thanks for the suggestion.</div><div><br></div><div>But if I offer authenticate after 100-200 records, the hacker just give up that cookie, ip and get a new ip to bypass the restriction.</div>
<div><br></div><div>Regards</div><div>Leigh</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 4 July 2014 10:51, John Bird <span dir="ltr"><<a href="mailto:johnkbird@paradise.net.nz" target="_blank">johnkbird@paradise.net.nz</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div dir="ltr">
<div style="FONT-SIZE:12pt;FONT-FAMILY:'Calibri';COLOR:#000000">
<div>Sounds to me like the data needs to be hidden from the web site in a
database with authentication to view records. You could easily allow
users to see say some eg 100-200 records even if not authenticated but not
more.</div>
<div> </div>
<div>There is a similar restriction in data for instance with electoral roles –
you can look up names but not scrape data – there the reasons are for privacy as
well as to protect their income as they sell data to certain users only.</div>
<div style="FONT-SIZE:small;TEXT-DECORATION:none;FONT-FAMILY:"Calibri";FONT-WEIGHT:normal;COLOR:#000000;FONT-STYLE:normal;DISPLAY:inline">
<div style="FONT:10pt tahoma">
<div> </div>
<div style="BACKGROUND:#f5f5f5">
<div><b>From:</b> <a title="leigh.wanstead@gmail.com" href="mailto:leigh.wanstead@gmail.com" target="_blank">Leigh Wanstead</a> </div>
<div><b>Sent:</b> Friday, July 4, 2014 10:39 AM</div><div><div class="h5">
<div><b>To:</b> <a title="delphi@listserver.123.net.nz" href="mailto:delphi@listserver.123.net.nz" target="_blank">NZ Borland Developers Group - Delphi
List</a> </div>
<div><b>Subject:</b> Re: [DUG] Work Wanted in Wellington</div></div></div></div></div>
<div> </div></div>
<div style="FONT-SIZE:small;TEXT-DECORATION:none;FONT-FAMILY:"Calibri";FONT-WEIGHT:normal;COLOR:#000000;FONT-STYLE:normal;DISPLAY:inline"><div><div class="h5">
<div dir="ltr">Hi David,
<div> </div>
<div>It is like amazon. Amazon does not require user name/password just browsing
the data.</div>
<div> </div>
<div>Regards</div>
<div>Leigh</div></div>
<div class="gmail_extra"><br><br>
<div class="gmail_quote">On 4 July 2014 10:29, David Brennan <span dir="ltr"><<a href="mailto:dugdavid@dbsolutions.co.nz" target="_blank">dugdavid@dbsolutions.co.nz</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="PADDING-LEFT:1ex;MARGIN:0px 0px 0px 0.8ex;BORDER-LEFT:#ccc 1px solid">
<div lang="EN-NZ" vlink="purple" link="blue">
<div>
<p class="MsoNormal"><span style="FONT-SIZE:11pt;FONT-FAMILY:"Calibri","sans-serif";COLOR:#1f497d">Sounds
unusual. So the company sells the data but doesn’t have a login system to
control who consumes the data?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="FONT-SIZE:11pt;FONT-FAMILY:"Calibri","sans-serif";COLOR:#1f497d"><u></u><u></u></span> </p>
<p class="MsoNormal"><span style="FONT-SIZE:11pt;FONT-FAMILY:"Calibri","sans-serif";COLOR:#1f497d">David.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="FONT-SIZE:11pt;FONT-FAMILY:"Calibri","sans-serif";COLOR:#1f497d"><u></u><u></u></span> </p>
<p class="MsoNormal"><b><span lang="EN-US" style="FONT-SIZE:10pt;FONT-FAMILY:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="FONT-SIZE:10pt;FONT-FAMILY:"Tahoma","sans-serif""> <a href="mailto:delphi-bounces@listserver.123.net.nz" target="_blank">delphi-bounces@listserver.123.net.nz</a> [mailto:<a href="mailto:delphi-bounces@listserver.123.net.nz" target="_blank">delphi-bounces@listserver.123.net.nz</a>] <b>On Behalf Of
</b>Leigh Wanstead<br><b>Sent:</b> Friday, 4 July 2014 10:16 a.m.</span></p>
<div><br><b>To:</b> NZ Borland Developers Group - Delphi
List<br><b>Subject:</b> Re: [DUG] Work Wanted in
Wellington<u></u><u></u></div>
<p class="MsoNormal"><u></u><u></u> </p>
<div>
<p class="MsoNormal">Hi Jolyon,<u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal"><u></u><u></u> </p></div>
<div>
<p class="MsoNormal">The company I work for is selling data. The data is the
income of the company.<u></u><u></u></p></div>
<div>
<p class="MsoNormal"><u></u><u></u> </p></div>
<div>
<p class="MsoNormal">Regards<u></u><u></u></p></div>
<div>
<p class="MsoNormal">Leigh<u></u><u></u></p></div></div></div></div>
<div>
<div>
<div>
<p class="MsoNormal" style="MARGIN-BOTTOM:12pt"><u></u><u></u> </p>
<div>
<p class="MsoNormal">On 4 July 2014 09:23, Jolyon Smith <<a href="mailto:jsmith@deltics.co.nz" target="_blank">jsmith@deltics.co.nz</a>>
wrote:<u></u><u></u></p>
<div>
<p class="MsoNormal">I don't understand this determination to make the hacker's
life difficult. Surely the objective is to address the impact on the
site for legitimate users ?<br><br>Contriving schemes to make the hacker's
life difficult is simply extending the problem domain into an irrelevant area
and increasing the complexity by orders of magnitude in order to protect
information that is public already - there is no mention of any attempt to
thwart site security, only scraping of publicly accessible URL's..<br><br>If
the intent is to disincentivise the hacker, simply denying them the ability to
scrape the site by detecting and blocking them will cause them inconvenience
enough. Even if it doesn't, as long as their activity is not impacting
on the legitimate operation of the site then the key objective is met - that
of maintaining site response for legit users.<br><br>Almost all of these
schemes to make the scrapers life miserable do also impact on the legitimate
user experience, loading up the server and the client browser processing with
overhead targeted at the scraper but imposed on ALL clients.<br><br><br>I can
see that the technical challenge of "beating" the hacker could be attractive,
but it seems to me to be an ultimately pointless and resource sapping "Arms
Race" that cannot ever really be won... even if you eventually drive the
scraper to give up entirely, burdensome counter-measures will themselves have
impacted on your site, defeating if not the whole object then certainly a
significant part of it, of getting rid of the scraper activity in the first
place.<br><br>Of course, if you can find counter-measures which do not impose
any such burden on legit users then you have the best of both worlds, but the
key need to be met is addressing the scraper by removing the impact on legit
users, not adding to it.<br><br><br>So, bringing it back to the original topic
- What makes a good developer ?<br><br>Another characteristic would be the
ability to remain focused on the key objective/user need, rather than being
drawn into a bottomless honey pot of technical challenge of limited/no direct
relevance to the problem at hand.<br><br>:)<u></u><u></u></p></div>
<div>
<p class="MsoNormal" style="MARGIN-BOTTOM:12pt"><u></u><u></u> </p>
<div>
<div>
<p class="MsoNormal">On 4 July 2014 09:05, Phil Scadden <<a href="mailto:p.scadden@gns.cri.nz" target="_blank">p.scadden@gns.cri.nz</a>>
wrote:<u></u><u></u></p></div>
<div>
<div>
<blockquote style="BORDER-TOP:medium none;BORDER-RIGHT:medium none;BORDER-BOTTOM:medium none;PADDING-BOTTOM:0cm;PADDING-TOP:0cm;PADDING-LEFT:6pt;MARGIN-LEFT:4.8pt;BORDER-LEFT:#cccccc 1pt solid;PADDING-RIGHT:0cm;MARGIN-RIGHT:0cm">
<p class="MsoNormal"><br>> Regarding to render the website in Javascript,
how are you going to<br>> stop the browser driven by script? The hacker
does not need to<br>> understand the javascript. All he need is just grab
dom element.<br>That would be true but very unlikely that hacker is using
browser. Too<br>slow. If you load the html with junk data and modify it with
js, it may<br>take the hacker a long time to notice they are using crap. But
I would<br>looking at detecting the hacker without a tip off in first place
and<br>then figure out ways to make life difficult.<br><br><br>--<br>Phil
Scadden, Senior Scientist GNS Science Ltd 764 Cumberland St,<br>Private Bag
1930, Dunedin, New Zealand Ph <a href="tel:%2B64%203%204799663" target="_blank">+64 3 4799663</a>, fax <a href="tel:%2B64%203%20477%205232" target="_blank">+64 3 477 5232</a><br><br>Notice: This email and any
attachments are confidential.<br>If received in error please destroy and
immediately notify us.<br>Do not copy or disclose the
contents.<br><br>_______________________________________________<br>NZ
Borland Developers Group - Delphi mailing list<br>Post: <a href="mailto:delphi@listserver.123.net.nz" target="_blank">delphi@listserver.123.net.nz</a><br>Admin: <a href="http://delphi.org.nz/mailman/listinfo/delphi" target="_blank">http://delphi.org.nz/mailman/listinfo/delphi</a><br>
Unsubscribe:
send an email to <a href="mailto:delphi-request@listserver.123.net.nz" target="_blank">delphi-request@listserver.123.net.nz</a> with Subject:
unsubscribe<u></u><u></u></p></blockquote></div></div></div>
<p class="MsoNormal"><u></u><u></u> </p></div>
<p class="MsoNormal"><br>_______________________________________________<br>NZ
Borland Developers Group - Delphi mailing list<br>Post: <a href="mailto:delphi@listserver.123.net.nz" target="_blank">delphi@listserver.123.net.nz</a><br>Admin: <a href="http://delphi.org.nz/mailman/listinfo/delphi" target="_blank">http://delphi.org.nz/mailman/listinfo/delphi</a><br>
Unsubscribe:
send an email to <a href="mailto:delphi-request@listserver.123.net.nz" target="_blank">delphi-request@listserver.123.net.nz</a> with Subject:
unsubscribe<u></u><u></u></p></div>
<p class="MsoNormal"><u></u><u></u> </p></div></div></div></div></div><br>_______________________________________________<br>NZ
Borland Developers Group - Delphi mailing list<br>Post: <a href="mailto:delphi@listserver.123.net.nz" target="_blank">delphi@listserver.123.net.nz</a><br>Admin:
<a href="http://delphi.org.nz/mailman/listinfo/delphi" target="_blank">http://delphi.org.nz/mailman/listinfo/delphi</a><br>Unsubscribe:
send an email to <a href="mailto:delphi-request@listserver.123.net.nz" target="_blank">delphi-request@listserver.123.net.nz</a>
with Subject: unsubscribe<br></blockquote></div>
<div> </div></div>
</div></div><p>
</p><hr><div class="">
_______________________________________________<br>NZ Borland Developers Group -
Delphi mailing list<br>Post: <a href="mailto:delphi@listserver.123.net.nz" target="_blank">delphi@listserver.123.net.nz</a><br>Admin:
<a href="http://delphi.org.nz/mailman/listinfo/delphi" target="_blank">http://delphi.org.nz/mailman/listinfo/delphi</a><br>Unsubscribe: send an email to
<a href="mailto:delphi-request@listserver.123.net.nz" target="_blank">delphi-request@listserver.123.net.nz</a> with Subject:
unsubscribe</div><p></p></div></div></div></div>
<br>_______________________________________________<br>
NZ Borland Developers Group - Delphi mailing list<br>
Post: <a href="mailto:delphi@listserver.123.net.nz">delphi@listserver.123.net.nz</a><br>
Admin: <a href="http://delphi.org.nz/mailman/listinfo/delphi" target="_blank">http://delphi.org.nz/mailman/listinfo/delphi</a><br>
Unsubscribe: send an email to <a href="mailto:delphi-request@listserver.123.net.nz">delphi-request@listserver.123.net.nz</a> with Subject: unsubscribe<br></blockquote></div><br></div>