[DUG] SOAP pack size
Robert martin
rob at chreos.co.nz
Fri Jul 18 15:14:43 NZST 2008
Hi
Might look into it. At the moment the XML creation performance is not a
problem.
Kyley Harris wrote:
> What are you using to create the XML..
>
> I use a library called NativeXML which is quick as XML goes (after
> optimising their code a bit )
>
> XML is sluggish to create...
>
> The secret to best performance is to hand write your XML creator with
> a TSTringlist because XML is dead simple to create, and then use a
> proper XML parser to process it..
>
> Things that take 1 second using the proper XML object to read and
> write, can take about 20ms to generate with a stringlist and basic XML
> code generator.
>
> On Fri, Jul 18, 2008 at 12:27 PM, Robert martin <rob at chreos.co.nz
> <mailto:rob at chreos.co.nz>> wrote:
>
> Hi Guys
>
> Just to let you know that I have this working GREAT now. I have
> surfaced an 'OnBeforeExecuteStream' event in a customised
> THTTPRIO. In
> this I take the output stream (The 1mb XML file) and zip it as
> suggested. On the SOAP server I unzip it on the OnbeforeDispatch.
>
> This compresses files down to about 40kb. It is wonderful ! although
> the slowest part of the process seems to be creating the 1mb XML file.
>
> Thanks everyone for your help.
>
> Rob
>
>
> Kyley Harris wrote:
> > well to give you an example..
> >
> > I have an object that represents a customer with about 20 fields of
> > information... names addresses etc.. I can convert about 5000 to an
> > xml structure that compressed to a size that takes a couple of
> seconds
> > on dial up... I get roughly each 1mb of XML transfers using my
> > protocol at around 40-60k of actual transferred size using a zip
> > compression... for you, using a webservice, you would need to add an
> > IIS or APache compression layer into the server, which only works if
> > client and server both have it...
> >
> > if you don't need to send 300 at a time.. (i wonder is each sales
> > lines items less than 10 etc) if you might be better to break
> the job
> > down into a series of sends, and a batch completion...
> >
> > How I would do it is to send a SOAP packet which then has one field
> > inside called DATA.. and data is the compressed zipped transaction
> > data.. your webservice can open the soap packet and unzip the DATA
> > field with zlib. you will find that your 5mb XML transactions are
> > probably only about 100-200k of filesize zipped.
> >
> > On Thu, Jul 17, 2008 at 2:30 PM, Robert martin <rob at chreos.co.nz
> <mailto:rob at chreos.co.nz>
> > <mailto:rob at chreos.co.nz <mailto:rob at chreos.co.nz>>> wrote:
> >
> > Hi
> >
> > I may be doing this completely wrong but I am sending a set of
> > transaction objects, each represents one transaction in a
> system. It
> > has a set of sale lines and a set of payment lines. I am
> sending
> > about
> > 300 at a time. I used the word packet meaning 300 as
> opposed to a
> > TCP/IP packet. Sorry about the confusion.
> >
> > Re compression, how would I go about compressing data in
> this format?
> >
> > Thanks
> > Rob
> >
> >
> >
> > Kyley Harris wrote:
> > > what kind of data are you sending that ends up being 1MB in
> > size? How
> > > often are the packets being sent.. Thats a huge amount of
> traffic if
> > > its continuous usage..
> > > could you indicate what the size of your packet is compressed?
> > > compression might not even help...
> > >
> > >
> > >
> > >
> > >
> > > On Thu, Jul 17, 2008 at 1:39 PM, Robert martin
> <rob at chreos.co.nz <mailto:rob at chreos.co.nz>
> > <mailto:rob at chreos.co.nz <mailto:rob at chreos.co.nz>>
> > > <mailto:rob at chreos.co.nz <mailto:rob at chreos.co.nz>
> <mailto:rob at chreos.co.nz <mailto:rob at chreos.co.nz>>>> wrote:
> > >
> > > Hi
> > >
> > > We are using Apache. Will look into it. I think for
> the time
> > > being the
> > > data size is not too big an issue.
> > >
> > >
> > >
> > > Alister Christie wrote:
> > > > I don't know if this helps much but some web servers
> > (probably just
> > > > Apache) support zip compression downloads - they might
> > support it
> > > > uploads as well - although I have no idea how.
> > > >
> > > > Alister Christie
> > > > Computers for People
> > > > Ph: 04 471 1849 Fax: 04 471 1266
> > > > http://www.salespartner.co.nz
> > > > PO Box 13085
> > > > Johnsonville
> > > > Wellington
> > > >
> > > >
> > > >
> > > > Robert martin wrote:
> > > >
> > > >> Hi
> > > >>
> > > >> We have got the basis of a SOAP application up and
> > running. It is
> > > >> transferring some data from one machine to a web server
> > perfectly.
> > > >>
> > > >> However after sniffing the transaction it seems
> that each
> > > 'packet' of
> > > >> data we send is about 1MB. This is only about 1/3
> of the
> > data
> > > we plan
> > > >> to send and we anticipate receiving some data also.
> A small
> > > number of
> > > >> users will be on dial up. 1Mb would be ok but it seems
> > like we
> > > might
> > > >> end up around 5Mb. This is too much.
> > > >>
> > > >> I know SOAP is a bulky XML protocol but does anyone
> know
> > of ways to
> > > >> reduce the XML size (appart from encoding the data as a
> > binary
> > > file,
> > > >> compressing it and sending).
> > > >>
> > > >> We might have to scrap what we have done so far.
> But asside
> > > from the
> > > >> file size, the system works great. It would be a
> shame :-(
> > > >>
> > > >> Thanks
> > > >> Rob
> > > >> _______________________________________________
> > > >> NZ Borland Developers Group - Delphi mailing list
> > > >> Post: delphi at delphi.org.nz
> <mailto:delphi at delphi.org.nz> <mailto:delphi at delphi.org.nz
> <mailto:delphi at delphi.org.nz>>
> > <mailto:delphi at delphi.org.nz <mailto:delphi at delphi.org.nz>
> <mailto:delphi at delphi.org.nz <mailto:delphi at delphi.org.nz>>>
> > > >> Admin: http://delphi.org.nz/mailman/listinfo/delphi
> > > >> Unsubscribe: send an email to
> > delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>
> <mailto:delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>>
> > > <mailto:delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>
> > <mailto:delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>>> with Subject: unsubscribe
> > > >>
> > > >>
> > > >>
> > > >
> > > > _______________________________________________
> > > > NZ Borland Developers Group - Delphi mailing list
> > > > Post: delphi at delphi.org.nz
> <mailto:delphi at delphi.org.nz> <mailto:delphi at delphi.org.nz
> <mailto:delphi at delphi.org.nz>>
> > <mailto:delphi at delphi.org.nz <mailto:delphi at delphi.org.nz>
> <mailto:delphi at delphi.org.nz <mailto:delphi at delphi.org.nz>>>
> > > > Admin: http://delphi.org.nz/mailman/listinfo/delphi
> > > > Unsubscribe: send an email to
> delphi-request at delphi.org.nz <mailto:delphi-request at delphi.org.nz>
> > <mailto:delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>>
> > > <mailto:delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>
> > <mailto:delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>>> with Subject: unsubscribe
> > > >
> > > >
> > > _______________________________________________
> > > NZ Borland Developers Group - Delphi mailing list
> > > Post: delphi at delphi.org.nz
> <mailto:delphi at delphi.org.nz> <mailto:delphi at delphi.org.nz
> <mailto:delphi at delphi.org.nz>>
> > <mailto:delphi at delphi.org.nz <mailto:delphi at delphi.org.nz>
> <mailto:delphi at delphi.org.nz <mailto:delphi at delphi.org.nz>>>
> > > Admin: http://delphi.org.nz/mailman/listinfo/delphi
> > > Unsubscribe: send an email to
> delphi-request at delphi.org.nz <mailto:delphi-request at delphi.org.nz>
> > <mailto:delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>>
> > > <mailto:delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>
> > <mailto:delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>>> with Subject: unsubscribe
> > >
> > >
> > >
> > >
> > > --
> > > Kyley Harris
> > > Harris Software
> > > +64-21-671-821
> > >
> >
> ------------------------------------------------------------------------
> > >
> > > _______________________________________________
> > > NZ Borland Developers Group - Delphi mailing list
> > > Post: delphi at delphi.org.nz <mailto:delphi at delphi.org.nz>
> <mailto:delphi at delphi.org.nz <mailto:delphi at delphi.org.nz>>
> > > Admin: http://delphi.org.nz/mailman/listinfo/delphi
> > > Unsubscribe: send an email to delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>
> > <mailto:delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>> with Subject: unsubscribe
> > _______________________________________________
> > NZ Borland Developers Group - Delphi mailing list
> > Post: delphi at delphi.org.nz <mailto:delphi at delphi.org.nz>
> <mailto:delphi at delphi.org.nz <mailto:delphi at delphi.org.nz>>
> > Admin: http://delphi.org.nz/mailman/listinfo/delphi
> > Unsubscribe: send an email to delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>
> > <mailto:delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz>> with Subject: unsubscribe
> >
> >
> >
> >
> > --
> > Kyley Harris
> > Harris Software
> > +64-21-671-821
> >
> ------------------------------------------------------------------------
> >
> > _______________________________________________
> > NZ Borland Developers Group - Delphi mailing list
> > Post: delphi at delphi.org.nz <mailto:delphi at delphi.org.nz>
> > Admin: http://delphi.org.nz/mailman/listinfo/delphi
> > Unsubscribe: send an email to delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz> with Subject: unsubscribe
> _______________________________________________
> NZ Borland Developers Group - Delphi mailing list
> Post: delphi at delphi.org.nz <mailto:delphi at delphi.org.nz>
> Admin: http://delphi.org.nz/mailman/listinfo/delphi
> Unsubscribe: send an email to delphi-request at delphi.org.nz
> <mailto:delphi-request at delphi.org.nz> with Subject: unsubscribe
>
>
>
>
> --
> Kyley Harris
> Harris Software
> +64-21-671-821
> ------------------------------------------------------------------------
>
> _______________________________________________
> NZ Borland Developers Group - Delphi mailing list
> Post: delphi at delphi.org.nz
> Admin: http://delphi.org.nz/mailman/listinfo/delphi
> Unsubscribe: send an email to delphi-request at delphi.org.nz with Subject: unsubscribe
More information about the Delphi
mailing list