[DUG] DLL loading slowed by anti-virus

Tue Sep 22 23:49:44 NZST 2015

Some customer sites could have 15 DLL’s load at different times.

The 2 apps are always loaded together.  One loads the other one and they communicate via shared memory.  I won’t go into why it was done that way.

But I understand all you are saying, thanks for the code, and I’ll think about it.

Cheers.

From: delphi-bounces at listserver.123.net.nz [mailto:delphi-bounces at listserver.123.net.nz] On Behalf Of Jolyon Direnko-Smith
Sent: Tuesday, 22 September 2015 5:28 p.m.
To: NZ Borland Developers Group - Delphi List
Subject: Re: [DUG] DLL loading slowed by anti-virus

Hi Ross,

It might work.  sounds plausible and is easy enough to test to make sure.  But it seems crude and inefficient.  You said yourself that in any one customer (site?) only 2 or 3 of the 30 DLL's are actually used.

I don't know how the two apps involved are used, but you are adding 30x the DLL delay to an app that doesn't use any of the DLL's and if the app that does use the DLL(s) is started at the same time and the 1 DLL it needs is the 30th one to be loaded then it could still be the 2nd app which suffers the delay if it happens to load it's DLL before the "loader" app has gotten around to loading that particular one.

And then even assuming that DLL loading does work the way you expect/hope, this isn't guaranteed to always be the case - you are relying on a behaviour in the OS which, however unlikely you may think it is, may change.  Even if the OS behaviour never changes, if the first app is unloaded for any reason, you suddenly lose your "pre-loaded" DLL's and the only way to get them back is to via some operating procedure that instructs the user to start this other app that doesn't even use the DLL's.

Also, is there a use case where a new DLL might be installed while these apps are running ?  If so, this won't get the benefit of the pre-loading by the application unless and until that pre-loader app is restarted.

Or, replace a DLL ?  In which case, after closing the application which uses those DLL's, the user must now also close the other application which does not.  If they only shut down the application which uses the DLL, then replace a DLL, after restarting the application it will continue to be using the older version of that DLL previously (and still) loaded by the loader app.

It all seems very messy.  Which is why fundamentally I am wary of solutions that involve putting the "fix" at a remove from the problem, and which do not solve the problem directly but rely on side effects to achieve the desired outcome.  :)

So, if by "leave loaded permanently" you mean "load when first needed and retain for when needed again, switching between the loaded DLL's as required", then yes, that is what I mean.  :)

I don't see that it should be that tricky.

Even if you currently have all your DLL entry points in simple variables and don't currently have a facade, you presumably still have some function point at which a DLL is loaded (and any current DLL 'closed') which sets the entry point variables ?  So all you need to do is introduce the facade to manage the persisted function pointers and then fix-up your simple variables from the current "in context" library, before returning from your "Load" routine.

e.g. if you have a LoadDLL procedure which perhaps currently looks something like this (no error checking and not all declarations shown etc, for brevity/clarity):

    procedure LoadDLL( filename );

    begin

       if hDLL <> 0 then

          FreeLibrary(hDLL);

       hDLL := LoadLibrary(filename);

       fnFoo := GetProcAddress(hDLL, 'foo');

       fnBar := GetProcAddress(hDLL, 'bar');

    end;

then it becomes:

    procedure LoadDLL( filename );

    begin

       DLLManager.Load(filename);

       fnFoo := DLLManager.Foo;

       fnBar := DLLManager.Bar;

    end;

DLLManager is an instance of some class (which you need to code, obviously) which keeps all the ProcAddress-es for each DLL filename in some associative list and exposes only the "active" (i.e. most recently Load()-ed) function entry points via properties which you can then assign to your variables.

The DLLManager.Load() method loads the specified DLL file if required (the code for which is essentially already written in your current LoadDLL() method, as illustrated above).  The only difference being that the entry points are assigned to members of some record associated with that specific DLL filename, perhaps in a record type:

   PLoadedDLL = ^TLoadedDLL; 

   TLoadedDLL = record

      Handle: HMODULE;

      Filename: String;

      fnFoo: TFooFn;

      fnBar: TBarFn;

    end;

    fLoaded: array of TLoadedDLL;

    fActive: PLoadedDLL;

When loading a specific DLL filename, DLLManager iterates over the array looking for that filename.  If found, then make fActive reference that entry.  Otherwise extend the array by one, load the DLL, set the entry point and other members of the new TLoadedDLL record and make that new entry the fActive reference.

The entry point properties then use getters to simply return the current active entry points:

   function TDLLManager.get_Foo: TFooProc;

   begin

      result := fActive.fnFoo;

   end;

You could make the DLLManager set any existing function entry point variables directly, but I see this as the job of the existing "LoadDLL" method, since this is the connective "glue" that binds the self contained job of the DLLManager to the needs of your current application code.

However you go about updating those entry point variables, all the existing code that uses these variables to call the DLL entry points is blissfully oblivious to the more sophisticated DLL management going on - all they know (and need to know) is that there is a currently active DLL they can call.  All the work is neatly encapsulated within the DLLManager object.

Add some cleanup code to FreeLibrary() all entries in the fLoaded array when the app closes and job done: a clean, reliable solution applied directly where the problem lies.  :)

hth

--

Jolyon

On 22 September 2015 at 16:45, Ross Levis <ross at stationplaylist.com> wrote:

You mean leave the DLL loaded permanently and switch to it when required?  I thought of that but the code is quite tricky as it is in that app.  I was hoping to not modify that app and just load the DLL’s in another one.  I take it my original plan would work?

From: delphi-bounces at listserver.123.net.nz [mailto:delphi-bounces at listserver.123.net.nz] On Behalf Of Jolyon Direnko-Smith
Sent: Tuesday, 22 September 2015 3:24 p.m.
To: NZ Borland Developers Group - Delphi List
Subject: Re: [DUG] DLL loading slowed by anti-virus

Hi Ross,

Rather than have an application load 30 DLLS (none of which it uses) for the sake of another application which might use as few as just 2 of those 30 and only ever one at a time, why not simply cache each loaded DLL in the application that is actually using them ?

I would hope that your function references to the DLL exports are already maintained in a record or even behind a class facade, which would make a cache simplicity itself to implement.  If not, introducing such a facade should be straightforward enough.

With such a cache you still get the initial delay but only for the first use and only for the DLL's actually being used and it eliminates any question about (or possible future variance in) the module loading behaviour of the OS or any problems arising as a result of your "loader" application failing to have loaded at/by the required or expected time etc.

--

Jolyon

On 22 September 2015 at 14:42, Ross Levis <ross at stationplaylist.com> wrote:

My automation app loads 1 of perhaps 30 different 3rd party DLL’s depending on the job required.  It only has to do 1 job at a time and if the job changes (automatically), the current DLL is unloaded and the relevant one is loaded using LoadLibrary.  All the DLL’s have the same interface/functions.

It seems in recent times that the loading of a DLL is taking much too long for many customers, sometimes several seconds, which isn’t acceptable, and it appears to be related to anti-virus software scanning the DLL every time it loads, or at least the first time it has loaded on that day.  That seems to vary in different anti-virus software.

I need to remove that delay.  I would rather the main app not load every single DLL at startup as that would take a long time to load, and often only 2 or 3 of the DLL’s will ever be used by a customer.

There are 2 apps loaded at the same time and rather than altering the app that loads the DLL’s and does all the work, my thought was to have this 2nd app load all the DLL’s in the folder using a thread.

Am I correct that a 2nd app loading the same DLL will not actually load the DLL from the hard drive but just access it from RAM, so it should load very quickly?

Cheers,

Ross.

_______________________________________________
NZ Borland Developers Group - Delphi mailing list
Post: delphi at listserver.123.net.nz
Admin: http://delphi.org.nz/mailman/listinfo/delphi
Unsubscribe: send an email to delphi-request at listserver.123.net.nz with Subject: unsubscribe

_______________________________________________
NZ Borland Developers Group - Delphi mailing list
Post: delphi at listserver.123.net.nz
Admin: http://delphi.org.nz/mailman/listinfo/delphi
Unsubscribe: send an email to delphi-request at listserver.123.net.nz with Subject: unsubscribe

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://listserver.123.net.nz/pipermail/delphi/attachments/20150922/33b1eb4c/attachment-0001.html 