[DUG] Validating CDS files

[Matthew Comb]

Hi Todd,

The ClientDatasets initially get populated via a remote server, and it is
this process which in rare cases causes the corruption (e.g. customer on
flakey wireless lan or similar).

Longterm we will replace this mechanism so the dataset is populated server
side and send back with a hash which will maintain integrity.

For the moment, we are stuck with a mechanism which populates this cds
data and then writes to file.

Appending some metadata at this point is a valid option. Ill test and see
if it circumvents the issue. The main issue though is that I'm not sure if
the ClientDataset at this point knows that the data is corrupt, and
therefore we could be exporting a corrupt data chunk packed with some
metadata, which does not help.

Thats why we really need some protection on load.

Cheers,

Matt.



> Hi Matthew
>
> Are the CDS files being stored as disk files or in a database? How are
> they being corrupted? Faulty back up media? Perhaps you could add some
> meta-data to each file as it is saved.
>
> Todd.
>> The driver for the question, is that we have some application client
>> datasets which are put into a defaulted state if a corrupt cds file is
>> loaded.
>>
>> Yes with XML, we can just validate the XML, but we use the binary format
>> so that solution does not apply.
>>
>> At present we basically have two solutions.
>>
>> 1. Load into a test clientdataset as suggested by Alistair. This is a
>> valid solution but does add considerable load time into the startup.
>>
>> 2. Can load into application clientdatasets, and dispose and reload if
>> error encountered. This is ok also but does require additional loading
>> in
>> case of error.
>>
>> What I'm really after is a file level test to check that file should
>> even
>> be attempted. e.g. open file stream seek start and seek end and check a
>> couple of bytes... that type of thing.
>>
>> Matt.
>>
>>
>>
>>
>>> What does "validate" mean for a CDS file ?
>>>
>>> For XML it obviously/intuitively means "is a valid XML representation
>>> of a
>>> CDS file" but for a CDS file, if you dumped it from a client data set
>>> in
>>> the
>>> first place, what additional validation would be needed to ensure that
>>> it
>>> is
>>> a CDS file ?
>>>
>>>
>>> -----Original Message-----
>>> From: delphi-bounces at delphi.org.nz
>>> [mailto:delphi-bounces at delphi.org.nz]
>>> On
>>> Behalf Of Matthew Comb
>>> Sent: Monday, 17 January 2011 10:27
>>> To: NZ Borland Developers Group - Delphi List
>>> Subject: Re: [DUG] Validating CDS files
>>>
>>> Hey Cam, for space size reasons, we're using the CDS format not the XML
>>> format as the files get pretty large.
>>>
>>> Matt
>>>
>>>
>>>> If you have dumped the CDS as XML then you can validate it using XSD.
>>>> I
>>>> haven't come across a schema for it but it shouldn't be hard to make.
>>>>
>>>> Cameron Hart | Development Manager | Flow Software Limited
>>>> P: +64 9 476 3569 x910 | M: +64 21 222 3569 | E:
>>>> cameron.hart at flowsoftware.co.nz
>>>> PO Box 305-237, Triton Plaza, Auckland 0757, New Zealand |
>>>> www.flowsoftware.co.nz
>>>>
>>>> This message is intended for the addressee named above. It may contain
>>>> privileged or confidential information. If you are not the intended
>>>> recipient of this message you must not use, copy, distribute or
>>>> disclose
>>>> it to anyone.
>>>>
>>>> Please consider the environment before printing this email
>>>>
>>>> -----Original Message-----
>>>> From: delphi-bounces at delphi.org.nz
>>>> [mailto:delphi-bounces at delphi.org.nz]
>>>> On Behalf Of Matthew Comb
>>>> Sent: Monday, 17 January 2011 10:03 a.m.
>>>> To: NZ Borland Developers Group - Delphi List
>>>> Cc: NZ Borland Developers Group - Delphi List
>>>> Subject: [DUG] Validating CDS files
>>>>
>>>> Does anyone know if there is a way to Validate a CDS file (midas)
>>>> before
>>>> attempting to load it into the dataset ?
>>>>
>>>> Matt
>>>>
>>>> _______________________________________________
>>>> NZ Borland Developers Group - Delphi mailing list
>>>> Post: delphi at delphi.org.nz
>>>> Admin: http://delphi.org.nz/mailman/listinfo/delphi
>>>> Unsubscribe: send an email to delphi-request at delphi.org.nz with
>>>> Subject:
>>>> unsubscribe
>>>>
>>>> _______________________________________________
>>>> NZ Borland Developers Group - Delphi mailing list
>>>> Post: delphi at delphi.org.nz
>>>> Admin: http://delphi.org.nz/mailman/listinfo/delphi
>>>> Unsubscribe: send an email to delphi-request at delphi.org.nz with
>>>> Subject:
>>>> unsubscribe
>>>>
>>>>
>>>
>>> _______________________________________________
>>> NZ Borland Developers Group - Delphi mailing list
>>> Post: delphi at delphi.org.nz
>>> Admin: http://delphi.org.nz/mailman/listinfo/delphi
>>> Unsubscribe: send an email to delphi-request at delphi.org.nz with
>>> Subject:
>>> unsubscribe
>>>
>>> _______________________________________________
>>> NZ Borland Developers Group - Delphi mailing list
>>> Post: delphi at delphi.org.nz
>>> Admin: http://delphi.org.nz/mailman/listinfo/delphi
>>> Unsubscribe: send an email to delphi-request at delphi.org.nz with
>>> Subject:
>>> unsubscribe
>>>
>>>
>>
>> _______________________________________________
>> NZ Borland Developers Group - Delphi mailing list
>> Post: delphi at delphi.org.nz
>> Admin: http://delphi.org.nz/mailman/listinfo/delphi
>> Unsubscribe: send an email to delphi-request at delphi.org.nz with Subject:
>> unsubscribe
>>
>
> _______________________________________________
> NZ Borland Developers Group - Delphi mailing list
> Post: delphi at delphi.org.nz
> Admin: http://delphi.org.nz/mailman/listinfo/delphi
> Unsubscribe: send an email to delphi-request at delphi.org.nz with Subject:
> unsubscribe
>