Help! Strange Bug With Export + Dll File Write

[gkimsey]

I've run into a problem related to export sets and DLLs.

Basically, if I export a certain way after opening a project, then try to call a DLL which writes to a (different) file, the DLL write doesn't work. In fact, lots of other problems arise in DLLs because of this, not just the file writes, but that's the easiest thing to verify.

If, however, I simply call the DLL function, it runs and creates the file fine. Only if I export BEFORE calling the function does the problem arise. Again, the files are unrelated. The DLL is not writing to the same file that the export set is writing to.

I've attached a very small project which shows this, including a tiny DLL with one function that just writes some text to a file.

The project automatically loads the DLL. To see the problem, manually type the following two commands into the Command / Alert window:

beginexport(DebugExportSet)
fnSimpleDLL()

[/CODE]

This has to be done RIGHT AFTER OPENING THE CTL. If fnSimpleDLL() was run successfully already, the export doesn't seem to cause a problem for subsequent calls.

I included a sequence that shows this, but don't run it directly because somehow the timing of it means it doesn't always show the behavior.

To test that fnSimpleDLL() actually does something, [b]close and reopen[/b] the project and just type:

[CODE]
fnSimpleDLL()
[/CODE]

You should see "simpledll.txt" show up in the project directory (assuming you double clicked it from explorer. Side note: It's weird that DAQFactory's working directory is different depending on how you open a file. Tough to work around). By the way the DebugExportSet exports to C:\debug.csv, and the specific settings of the export set MAY MATTER.

I hope the procedure to reproduce the bug makes sense. I've verified this with two projects, different DLLs, different export sets, and different PCs. It's also holding me up on a much larger project so I'm hoping to get some feedback ASAP. In the meantime I'm trying to work around it by making sure I never export before running my DLL functions.

Here's the code for the DLL's function:

[CODE]
extern "C" SIMPLEDLL_API int fnSimpleDLL(void)
{
FILE * outfile = fopen("simpledll.txt", "a");
fprintf(outfile, "Hello\n");
fclose(outfile);
return 42;
}
[/CODE]

DLLProblem.zip

[AzeoTech]

I'm sorry but for security reasons we can not open a dll provided on the forum. We really need the source to the dll. We'll also need the dll source to help solve the problem too.

[gkimsey]

Attached is the source and project files for SimpleDLL. Compiled for Win32 under Visual Studio 2010 (Express OK).

Since posting the problem I've made a new DLL to replace DAQFactory's export in order to work around the problem. There were other benefits to this in my project (mainly being able to use ftell() to get the log file's size without having two separate file handles open).

I'm still interested in finding out the problem, though, so I can start using export sets again later.

SimpleDLL.zip

[AzeoTech]

If the files are different, then there probably is some issue with the way you are compiling the DLL. It's probably a library issue. Running the export set probably triggers the load of a certain Windows DLL to handle file handling, but that's a slightly different DLL than the one you are using, but when your code runs, it uses ours instead. This is because Windows has several DLLs that all do the same thing, but are linked in different ways and designed to run in different environments.

[gkimsey]

That makes sense. If DAQFactory has an old module loaded and

Any recommendations on tracking down which DLL might be the cause, or ways to avoid this? Since it's very basic file access (no unusual libraries) using the default DLL project under Visual Studio, it doesn't seem like it'd be easy to fix from my end. Because DAQFactory is the master process here, I don't have a debug environment with which to see what modules DAQFactory has loaded. If I did, I could cross-reference those with the modules opened by my DLL when run from a basic test bed, and see where a conflict might be.

[AzeoTech]

You can use Depends to figure out which modules are being used. Its most likely the C runtime library since that's the function causing the problems.

However, it also could be much simpler than that. It might be that you are using the wrong calling parameters into your DLL and that is causing some memory corruption which is causing things to be flaky in DAQFactory. You might be able to test this by commenting out all your file handling calls and just call a basically empty function. Alas, if it is wrong, the behavior may not be consistent.

[gkimsey]

I hadn't realized I could use Depends on DAQFactory -- thanks for that.

Looking into the CRT libraries, some more, I can't find any evidence to suggest that there would be any conflict between DAQFactory and my DLL with them, as all I'm doing is fopen, and not using the returned handle anywhere else.

The wrong calling parameters initially seems more likely (since I've certainly done that before and it's produced very wonky behavior), but it's a simple no-parameters-returns-int function, and it only has a problem if I export with DAQFactory first. None of my testing of this sample DLL has had trouble outside of that exact scenario.

Besides further chasing down the theoretical calling parameters problem (is returning int, compiled under Win32, a problem when interpretted as long by DAQFactory, also a 32-bit application?), I'm not sure what I can do without seeing the DAQFactory export source code, or at least being able to look at the file stream that's opened after the first export.

Probably a moot point since my workaround is still doing the job, but I feel like the sample DLL and tiny CTL project don't leave much room for error on my end, especially since the behavior persists on several PCs (including a 32-bit one running Windows XP, which should definitely eliminate any 64- vs 32-bit problems)

[AzeoTech]

I'll compile up your DLL and run it in a debugger with DAQFactory and see if anything pops up. The DLL extern() function hasn't changed in years and I haven't heard of any issues nor seen any myself, but maybe you hit on one that no one has before. More in a bit.

[gkimsey]

Okay, this became a big problem again, but I may have discovered a solution.

Up to this point on my main project I've been developing without hardware, simulating input signals with global variables and such. Now I'm integrating hardware with a LabJack U3-HV. I am not actually using the hardware in any sequences, but by simply enumerating channels (which do work, and behave as expected), now DLLs won't load. It seems similar to the above problem where a built-in DAQFactory functionality causes DLL problems if enabled, but the problem goes away when that functionality is disabled.

For kicks, I removed all the channels again and sure enough the DLLs can be loaded again. It seemed like the environment PATH that DAQFactory is using might somehow be changing based on whether the LabJack DLL was being loaded, and maybe that means DLLs in the CTL file's directory aren't in the PATH any more. Or it could be more related to the original problem that started this thread.

When I changed the DLLs to absolute paths instead of relative paths in the extern() calls, they worked again.

I think DAQFactory's search path is changing based on what is being loaded. To be clear, I've been opening the project by double-clicking the CTL file in order to use relative paths, which I mentioned in another thread was helpful for making the project portable and supporting multiple copies/versions of the project and all DLLs on the same system. It seems this is not a consistent method.

The reason I think this ties back to the original problem is that the DAQFactory export may have done something similar to the path, causing the DLL file writes to be operating under a different working directory than they should have been. However it doesn't actually SOLVE the original problem directly because the code...

extern "C" SIMPLEDLL_API int fnSimpleDLL(void)
{
  FILE * outfile = fopen("simpledll.txt", "a");
  fprintf(outfile, "Hello\n");
  fclose(outfile);
  return 42;
}
[/CODE]

...uses a file name path relative to the working directory which was probably changed to some arbitrary location based on DAQFactory's ExportSets functionality.

I'll work on porting everything to use absolute paths. If I'm right in my analysis, relative paths simply aren't supported in DAQFactory. It would be extremely valuable to be able to use relative paths (by not having the working directory change), or at least to *know* what the working directory was through a system function of some sort.

Please let me know if you have any suggestions for portability and allowing DLLs which use relative paths to work.

Thanks

[gkimsey]

False alarm in the previous post: Using absolute paths did not fix the DLL loading problem completely, just for one of three instances. So I'm still confused.

[AzeoTech]

I don't know. I compiled up your code, albeit under VC2008, then ran it in a debugger, loading the release version of DAQFactory and it functioned just fine. I changed the extern() in your document to the absolute path to the DLL, mostly for my benefit. I then tried it outside the debugger and again, just fine, though initially it wouldn't run the export set because without running DF as an Admin, it can't write to the root folder in Win7.

As for DLL loading paths, that's all determined by Windows.

My recommendation is this:

1) compile your DLL in Debug then run it in a debugger, loading DAQFactory as the executable. You won't be able to set a breakpoint until you run the extern() call since the DLL doesn't load until then.

2) determine if its a DLL loading issue or file issue. That will be pretty obvious if you do #1

3) if its a file issue, add some error handling and figure out what the actual error is from fopen().

[gkimsey]

OK,

I think I've resolved the problem, or at least worked around it for the time being. Essentially I discovered that while my own DLL was being found, its own dependency DLLs couldn't be found. DAQFactory's error is ambiguous here, just saying the top level DLL couldn't be loaded (can you pass the WINAPI error instead or in addition?). At the time my DLL was loaded, the Path environment variable is either blank or just very wrong in one way or another. I have not found out who the culprit of this is (LabJack DLL or DAQFactory).

Anyway, by using Delayed Loading of all the dependency DLLs (a linker option), I essentially deferred the file search to later on when my DLL's individual functions were actually called. At this time, the Path variable is "sane" and everything works fine.

My current assumption that the Path is being screwed up still seems like a valid explanation for the problem that started this thread (export + file write) and the problem that I just resolved (Use of LabJack Channels in DAQFactory project + custom DLL load with secondary dependencies). My understanding is that a process can change its own environment variables (e.g. Path) and those will get passed to child threads and processes (like the DLLs when loaded), but obviously doesn't change it for complete other processes.

All that said, I was able to reproduce the second problem (but not the first) with my own application using LoadLibrary and GetProcAddress, so it's certainly possible I'm just missing something about how the loading works, but all the same I still didn't have any of these issues until I added exporting or LabJack channels to my DAQFactory project.

Anyway, problem is worked around for now. For others who may encounter similar problems: try delayed loading of dependent DLLs if relevant.

I still have no legitimate solution for the original export problem that started this thread (I'm using my own DLL to export now).

[gkimsey]

Update on this: The DELAYLOAD linker option is still working around the problem on Windows 7, but after moving to Windows 8 it causes a crash. I've verified that the error is that it can't find the module, and I'm fairly certain at this point that it's the delayed load causing a more catastrophic error (instead of DAQFactory catching the problem).

Not sure how I'm going to get around this yet, but it all still goes back to DLLs not being found despite being in the same directory as the CTL file, so any ideas are welcome.

[gkimsey]

More updates: It occurred to me after typing up the last bit that on my Windows 7 machine I have the CTL file's directory added to the system PATH (which I'm convinced shouldn't be necessary, but apparently I'm wrong), and on the Windows 8 machine I didn't. Adding it fixed the crash with delayed load.

Hope this helps somebody. I don't consider it a great solution but it's a way to force Windows to use its full search path.

[AzeoTech]

You should put the secondary DLL's in the windows system folder. Actually, you probably should put the primary one there too. There are security reasons for this as well as the simple workaround to your issue.