[DUG] Validating CDS files

[Todd]

Hi Matthew

It sounds like comparing a MD5 hash of the CDS file prior to sending and 
subsequent to receiving the data stream would provide a sufficient check 
on its integrity.

Todd.
> Hi Todd,
>
> The ClientDatasets initially get populated via a remote server, and it is
> this process which in rare cases causes the corruption (e.g. customer on
> flakey wireless lan or similar).
>
> Longterm we will replace this mechanism so the dataset is populated server
> side and send back with a hash which will maintain integrity.
>
> For the moment, we are stuck with a mechanism which populates this cds
> data and then writes to file.
>
> Appending some metadata at this point is a valid option. Ill test and see
> if it circumvents the issue. The main issue though is that I'm not sure if
> the ClientDataset at this point knows that the data is corrupt, and
> therefore we could be exporting a corrupt data chunk packed with some
> metadata, which does not help.
>
> Thats why we really need some protection on load.
>
> Cheers,
>
> Matt.
>
>
>
>    
>> Hi Matthew
>>
>> Are the CDS files being stored as disk files or in a database? How are
>> they being corrupted? Faulty back up media? Perhaps you could add some
>> meta-data to each file as it is saved.
>>
>> Todd.
>>      
>>> The driver for the question, is that we have some application client
>>> datasets which are put into a defaulted state if a corrupt cds file is
>>> loaded.
>>>
>>> Yes with XML, we can just validate the XML, but we use the binary format
>>> so that solution does not apply.
>>>
>>> At present we basically have two solutions.
>>>
>>> 1. Load into a test clientdataset as suggested by Alistair. This is a
>>> valid solution but does add considerable load time into the startup.
>>>
>>> 2. Can load into application clientdatasets, and dispose and reload if
>>> error encountered. This is ok also but does require additional loading
>>> in
>>> case of error.
>>>
>>> What I'm really after is a file level test to check that file should
>>> even
>>> be attempted. e.g. open file stream seek start and seek end and check a
>>> couple of bytes... that type of thing.
>>>
>>> Matt.
>>>
>>>
>>>
>>>
>>>        
>>>> What does "validate" mean for a CDS file ?
>>>>
>>>> For XML it obviously/intuitively means "is a valid XML representation
>>>> of a
>>>> CDS file" but for a CDS file, if you dumped it from a client data set
>>>> in
>>>> the
>>>> first place, what additional validation would be needed to ensure that
>>>> it
>>>> is
>>>> a CDS file ?
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: delphi-bounces at delphi.org.nz
>>>> [mailto:delphi-bounces at delphi.org.nz]
>>>> On
>>>> Behalf Of Matthew Comb
>>>> Sent: Monday, 17 January 2011 10:27
>>>> To: NZ Borland Developers Group - Delphi List
>>>> Subject: Re: [DUG] Validating CDS files
>>>>
>>>> Hey Cam, for space size reasons, we're using the CDS format not the XML
>>>> format as the files get pretty large.
>>>>
>>>> Matt
>>>>
>>>>
>>>>          
>>>>> If you have dumped the CDS as XML then you can validate it using XSD.
>>>>> I
>>>>> haven't come across a schema for it but it shouldn't be hard to make.
>>>>>
>>>>> Cameron Hart | Development Manager | Flow Software Limited
>>>>> P: +64 9 476 3569 x910 | M: +64 21 222 3569 | E:
>>>>> cameron.hart at flowsoftware.co.nz
>>>>> PO Box 305-237, Triton Plaza, Auckland 0757, New Zealand |
>>>>> www.flowsoftware.co.nz
>>>>>
>>>>> This message is intended for the addressee named above. It may contain
>>>>> privileged or confidential information. If you are not the intended
>>>>> recipient of this message you must not use, copy, distribute or
>>>>> disclose
>>>>> it to anyone.
>>>>>
>>>>> Please consider the environment before printing this email
>>>>>
>>>>> -----Original Message-----
>>>>> From: delphi-bounces at delphi.org.nz
>>>>> [mailto:delphi-bounces at delphi.org.nz]
>>>>> On Behalf Of Matthew Comb
>>>>> Sent: Monday, 17 January 2011 10:03 a.m.
>>>>> To: NZ Borland Developers Group - Delphi List
>>>>> Cc: NZ Borland Developers Group - Delphi List
>>>>> Subject: [DUG] Validating CDS files
>>>>>
>>>>> Does anyone know if there is a way to Validate a CDS file (midas)
>>>>> before
>>>>> attempting to load it into the dataset ?
>>>>>
>>>>> Matt
>>>>>
>>>>> _______________________________________________
>>>>> NZ Borland Developers Group - Delphi mailing list
>>>>> Post: delphi at delphi.org.nz
>>>>> Admin: http://delphi.org.nz/mailman/listinfo/delphi
>>>>> Unsubscribe: send an email to delphi-request at delphi.org.nz with
>>>>> Subject:
>>>>> unsubscribe
>>>>>
>>>>> _______________________________________________
>>>>> NZ Borland Developers Group - Delphi mailing list
>>>>> Post: delphi at delphi.org.nz
>>>>> Admin: http://delphi.org.nz/mailman/listinfo/delphi
>>>>> Unsubscribe: send an email to delphi-request at delphi.org.nz with
>>>>> Subject:
>>>>> unsubscribe
>>>>>
>>>>>
>>>>>            
>>>> _______________________________________________
>>>> NZ Borland Developers Group - Delphi mailing list
>>>> Post: delphi at delphi.org.nz
>>>> Admin: http://delphi.org.nz/mailman/listinfo/delphi
>>>> Unsubscribe: send an email to delphi-request at delphi.org.nz with
>>>> Subject:
>>>> unsubscribe
>>>>
>>>> _______________________________________________
>>>> NZ Borland Developers Group - Delphi mailing list
>>>> Post: delphi at delphi.org.nz
>>>> Admin: http://delphi.org.nz/mailman/listinfo/delphi
>>>> Unsubscribe: send an email to delphi-request at delphi.org.nz with
>>>> Subject:
>>>> unsubscribe
>>>>
>>>>
>>>>          
>>> _______________________________________________
>>> NZ Borland Developers Group - Delphi mailing list
>>> Post: delphi at delphi.org.nz
>>> Admin: http://delphi.org.nz/mailman/listinfo/delphi
>>> Unsubscribe: send an email to delphi-request at delphi.org.nz with Subject:
>>> unsubscribe
>>>
>>>        
>> _______________________________________________
>> NZ Borland Developers Group - Delphi mailing list
>> Post: delphi at delphi.org.nz
>> Admin: http://delphi.org.nz/mailman/listinfo/delphi
>> Unsubscribe: send an email to delphi-request at delphi.org.nz with Subject:
>> unsubscribe
>>
>>      
>
> _______________________________________________
> NZ Borland Developers Group - Delphi mailing list
> Post: delphi at delphi.org.nz
> Admin: http://delphi.org.nz/mailman/listinfo/delphi
> Unsubscribe: send an email to delphi-request at delphi.org.nz with Subject: unsubscribe
>