Bradleys DasBlog! - Technical

VB6 - file format no longer supported

chris.bradley — Fri, 30 Apr 2010 21:12:04 GMT

Had an issue all of a sudden today opening a vb6 project we use all the time. It gave a cryptic message about a file format no longer being supported.

p.It turned out I had to go delete a corrupted oca file from teh vb6.exe folder. Best thing to do is crack open process explorer to see what file it's blowing up on and then go delete that OCA. Next time you add the reference it will get regenerated and then things are great. Where were ya on this one google?

ToolStripTextBox Memory Leaks

chris.bradley — Mon, 05 Apr 2010 21:28:57 GMT

I recently downloaded a copy of a new tool from red-gage called Ant Memory Profiler. Ant Memory Profiler is an application that allows you to do memory usage analysis on dot net apps to uncover how your app consumes memory.

During the course of trying out this tool we uncovered an interesting memory leak in the dot net framework ToolStrip control. Our symptom of this was that one of our application forms was being held in memory long after it should have been garbage collected. After a quick tour through google I found a some references on why this is occurring and how to ensure things get cleaned up properly.

Here’s how we add the cleanup code to the close event of the form…

Protected Overrides Sub OnClosed(ByVal e As System.EventArgs)

MyBase.OnClosed(e)

'note, this call is still required as it cleans up one set of extra handlers

reportViewer.Toolbar.Visible = False

'this is the really ugly stuff though, let our cleanup class take care of things

Dim gc2 As New ToolStripGarbageCollector

gc2.RemoveHandlers(reportViewer)

End Sub

And heres the meat behind the cleanup class. Don’t ask what this does, it’s pretty much a copy and paste I found from some newsgroups.

Public Class ToolStripGarbageCollector

Private Const EVENTHANDLER_ON_USER_PREFERENCE_CHANGED As String = "OnUserPreferenceChanged"

Private Const LIST_HANDLERS As String = "_handlers"

Private Const ON_USER_PREFERENCE_CHANGED_EVENT As String = "OnUserPreferenceChangedEvent"

Private Const SYSTEM_EVENT_INVOKE_INFO As String = "SystemEventInvokeInfo"

Private Const TARGET_DELEGATE As String = "_delegate"

Public Sub RemoveHandlers(ByVal ctrl As Control)

NavigateControls(ctrl)

End Sub

Public Sub NavigateControls(ByVal ctrl As Control)

Console.WriteLine(ctrl.Name & "-" & ctrl.GetType.FullName)

If ctrl.GetType.FullName = "System.Windows.Forms.ToolStripTextBox+ToolStripTextBoxControl" Then

RemoveUpcHandler(ctrl)

ElseIf ctrl.GetType.FullName = "System.Windows.Forms.ToolStrip" Then

RemoveUpcHandler(ctrl)

End If

For Each c As Control In ctrl.Controls

NavigateControls(c)

End Sub

Private Sub RemoveUpcHandler(ByVal ctrl As Control)

'create the delegate to OnUserPreferenceChanged

Dim d As [Delegate] = [Delegate].CreateDelegate(GetType(Microsoft.Win32.UserPreferenceChangedEventHandler), ctrl, EVENTHANDLER_ON_USER_PREFERENCE_CHANGED)

Dim handlers As Object = GetType(Microsoft.Win32.SystemEvents).GetField("_handlers", Reflection.BindingFlags.NonPublic Or Reflection.BindingFlags.Static).GetValue(Nothing)

Dim upcHandler As Object = GetType(Microsoft.Win32.SystemEvents).GetField(ON_USER_PREFERENCE_CHANGED_EVENT, Reflection.BindingFlags.NonPublic Or Reflection.BindingFlags.Static).GetValue(Nothing)

'get a SystemEventInvokeInfo type

Dim systemEventInvokeInfo As Object = GetType(Microsoft.Win32.SystemEvents).GetNestedType(SYSTEM_EVENT_INVOKE_INFO, Reflection.BindingFlags.NonPublic Or Reflection.BindingFlags.Instance)

'get the SystemEventInvokeInfo list for the UserPreferenceChangedEvent

Dim upcHandlerList As IList = CType(CType(handlers, IDictionary).Item(upcHandler), IList)

'initialize a target count

Dim targetCount As Integer = 0

Dim i As Integer = 0

'loop

While i < upcHandlerList.Count

systemEventInvokeInfo = upcHandlerList(i)

'get the SystemEventInvokeInfo._delegate field

Dim target As [Delegate] = CType(systemEventInvokeInfo.GetType().GetField(TARGET_DELEGATE, Reflection.BindingFlags.NonPublic Or Reflection.BindingFlags.Instance).GetValue(systemEventInvokeInfo), [Delegate])

'eval

If target.Target Is d.Target Then

'increment on positive ID

targetCount += 1

End If

i += 1

End While

'remove the handlers

For i = 1 To targetCount

RemoveHandler Microsoft.Win32.SystemEvents.UserPreferenceChanged, CType(d, Microsoft.Win32.UserPreferenceChangedEventHandler)

End Sub

End Class

AppDomains in Windows Services

chris.bradley — Sat, 27 Feb 2010 16:00:42 GMT

I've been meaning to write down something on this for a couple weeks now but have been having a tough time formulating a nice message on this. The short version is be very carefull doing this. I believe that somehow AppDomain's rely on something out of the Windows User Profile, and over time in a Windows Service when you're loading and unloading AppDomain's stuff MAY get screwed up.

We had a scheduler app which basically used AppDomain's as an isolation mechanism for controlling when and how each scheduled component was called and then disposed. Code looked something like this...

for each s as ScheduledComponent in myList
Dim d as new AppDomain
try
dim i as IExecutableInstance = d.CreateInstance("LaunchPad")
i.Execute
finally
AppDomain.Unload(d)
end try
next

Over time we'd eventually get into a state where the entire scheduler service which just cause a windows segmentation fault and shut down. No nice dot net exception or anything. Like I said, our best guess is that something gets out of sync between the windows profile and whatever exists in a dot net AppDomain.

Anyway, our solution was to create a wrapper layer which basically called each scheduled component as a process instead. This seems to be working very reliably ever since.

Windows 7 / Internet Explorer 8 Javascript

chris.bradley — Tue, 26 Jan 2010 15:39:45 GMT

We were having some problems on our development machines at work as soon as we upgraded to Windows 7 and IE8. The problem first manifested itself as a javascript "client out of memory" error while using Outlook Web Access (OWA) but it seemed like randomly we'd have javascript errors while navigating webpages.

Our workaround was to switch over to Google Chrome but then you start finding all the things that don't work with that so we just wanted to get this figured out.

We ended up working microsoft support on this one and long story short, it looks like one of our development tools is overwriting one of the IE classes. They had us re-register a couple classes and now it's working awesome. Here's the registery changes required...

[HKEY_CLASSES_ROOT\Interface\{79EAC9C5-BAF9-11CE-8C82-00AA004BA90B}]
@="IHlinkFrame"

[HKEY_CLASSES_ROOT\Interface\{79EAC9C5-BAF9-11CE-8C82-00AA004BA90B}\NumMethods]
@="8"

[HKEY_CLASSES_ROOT\Interface\{79EAC9C5-BAF9-11CE-8C82-00AA004BA90B}\ProxyStubClsid32]
@="{A4A1A128-768F-41E0-BF75-E4FDDD701CBA}"

BindingSource "Object reference not set to an instance of an object"

chris.bradley — Fri, 14 Aug 2009 15:35:42 GMT

We've had some problems with the Binding Source component in our Windows Forms development and I wanted to jot down a note on here in case anyone else comes across this.

The problem we were having was an IDE runtime error which looks like a NullReferenceException, the specific message says "Object reference not set to an instance of an object".

Apparently this has something to do with the .datasource files that are auto-generated by visual studio for holding "default" values. You can remove these from the My Project folder and then the error goes away. Not sure why these are even required since you can still build without them.

Active Reports Viewer IndexOutOfRangeException

chris.bradley — Thu, 07 May 2009 02:10:29 GMT

We’ve been fighting an intermittent error with our report viewer for the last couple years that’s been just terrible to track down and I think we finally made some progress today. We use a reporting tool called Data Dynamics Active Reports to generate our Screen and Paper reports. Their tool is basically a set of Dot Net assemblies that allow you to design and code a component called an Active Report.

At runtime, you execute this Active Report and then pass it into another component called an Active Report Viewer for displaying to the user. We found occasionally our users would get stuck in an infinite loop of error messages while displaying a report. Exceptions would pop up looking like this…

(IndexOutOfRangeException) Index was outside the bounds of the array.

at DataDynamics.ActiveReports.Viewer.-0011.-3cbd()

at DataDynamics.ActiveReports.Viewer.-dabf.-31a1()

at DataDynamics.ActiveReports.Viewer.-b212.-afcd()

We could never recreate this on demand, but probably five times a month we would get a hundred error messages coming in from a client site like this. Eventually the client would give up pressing “OK” on our error handler, and kill the process.

When we’ve tried to troubleshoot this in the past it’s been extremely difficult because Data Dynamics obfuscates their code so it can’t be decompiled. Obfuscation basically takes design-time code, and mixes it up so that it’s unreadable, hence the funky function names above. Unfortunately this gives you no sense of what’s actually happening in their app.

A little bit of digging on their support forums today did find a link to a similar issue by another user - http://www.datadynamics.com/forums/85506/ShowPost.aspx

Basically what they’d found was that in a Terminal Services environment, their users would click their “Print” button multiple times – the old adage, keep pressing the button until something happens. What this causes in some instances is for the second “click” to get handled by the newly created Active Report Viewer, even though it’s just in the process of being displayed. Now, if the Viewer is in a funny spot, it can try and interact with this click event while loading the document – that’s what seems to cause the issue.

Once we had a sense of what was causing the issue, it’s pretty simple to disable the ReportViewer until the report has generated its first page and sure enough this causes the issue to go away.

Source Code Version Control

chris.bradley — Tue, 24 Mar 2009 05:53:54 GMT

One area of concern that I come up against quite often is the issue of source code version control. At its most basic layer this is nothing more than being able to control what a version of your software consists of... version A versus version B, that sort of thing. This model works great and is well documented on smaller isolated applications but tends to blur a little when working with larger systems. Specifically we’ve had issues managing system dependencies, as well as change requirements on a larger application consisting of about roughly 50 assemblies. I’ll spend just a few minutes here putting pen to paper on how we’ve structured our development tree to accommodate this.

We’ll start at a pretty simple level; each major release is placed in a named folder based on the version name. Our first branch within that folder is named Major.Minor.Zero, Major and Minor are related to the version name while the Zero indicated this is the opening development branch. Here’s how that looks when we start a new major release... our friendly version name is Jupiler and the Major.Minor associated with this is 5.1.

Notice that within the 5.1.0 folder we have a number of sub-folders. These are what we call modules and basically relate to a VS.Net solution; each one consists of roughly 5-6 projects and can be thought of as a sub-system. These sub-systems collaborate to provide our applications functionality.

Now, when we release our major version, Jupiler, we’ll typically LOCK the 5.1.0 folder so that no further changes can be made. This code is set in stone and we want to force all new changes to be done in an isolated environment until they’re tested and ready to go. Invariable though, something critical comes up during rollout and we need to make code changes to address those concerns. When this situation arises we BRANCH into a new folder called Major.Minor.One. This code folder consists of all the new development changes required to add what we call a Service Pack to the release. Here’s how this looks.

One interesting aspect of this layout is that we only branch the modules that require changes. These Service Pack’s generally don’t require a lot of wholesale change; they`re usually small isolated bugs/features which can be addressed in a single module. You can see in the above diagram that our 5.1.1 only has changes for Accounting, all the other modules are identical to how they were developed in 5.1.0. This gives us a few key benefits.

1. It limits the amount of open source code.

2. It makes it easier to identify what code is actively being changed for a service pack.

3. It leaves the original branch open so that it can be reviewed/modified for OTHER unanticipated issues.

Final rollout of the new changes is done by releasing 5.1.1 of Accounting with all the previous versions of the other modules.

MSTSC Username/Password

chris.bradley — Fri, 06 Mar 2009 21:50:17 GMT

Not that this is necessarily a good idea. But after way too much time googling this, I think I'll clear up the fact that it IS possible call the Microsoft RDP client with a username and password parameter.

Now, everyone out there will say, yeah - we know that. You key in the username and password and if you check off the "remember credentials" checkbox it will store that data.

Well the trick is that this data is stored in a place that's not easily accessed -Windows Stored UserNames and Passwords. This area is basically a secured area of windows where passwords are stored for remote connections. You can access this by going Control Panel\User Accounts\Manager your network passwords. Here's a screen snap.

Our management infrastructure tracks this username and password in a seperate secured database, and then when we call MSTSC.exe from our software we want to pass this information. Now, a little more information on how you can pass parameters to MSTSC is in order. When you call MSTSC you can pass as a parameter a .RDP file which contains information about how you want to connect to the remote computer. Same idea as every other document type in Windows, a .RDP file is associated with MSTSC and is used to contain information for that application. Fortunately the .RDP files are basically key value pairs. Check it out...

screen mode id:i:1
desktopwidth:i:1024
desktopheight:i:768
session bpp:i:16
winposstr:s:0,1,526,61,1566,865
full address:s:SomeIpAddress
compression:i:1
keyboardhook:i:2
audiomode:i:0
redirectdrives:i:0
redirectprinters:i:1
redirectcomports:i:0
redirectsmartcards:i:1
displayconnectionbar:i:0
autoreconnection enabled:i:1
alternate shell:s:
shell working directory:s:
disable wallpaper:i:0
disable full window drag:i:1
disable menu anims:i:1
disable themes:i:0
disable cursor setting:i:0
bitmapcachepersistenable:i:1
allow desktop composition:i:1
allow font smoothing:i:0
redirectclipboard:i:1
redirectposdevices:i:0
authentication level:i:2
prompt for credentials:i:0
negotiate security layer:i:1
remoteapplicationmode:i:0
gatewayhostname:s:
gatewayusagemethod:i:4
gatewaycredentialssource:i:4
gatewayprofileusagemethod:i:0
promptcredentialonce:i:1
drivestoredirect:s:

The imporant pieces here is the "full address" line... this points to the server we're connecting to.

Well... it turns out you can also pass parameters for domain, username, and password. You just need to add them to the bottom of your .rdp file like this

domain:s:MyDomain
username:s:MyUserName
password 51:b:MyPassword

But wait a second, saving a password in plain text in a .rdp file sounds like just about the worst security hole you could use! And you're right - you can't actually pass the password parameter as straight text. It needs to be encrypted binary.

(As an aside, that's what the second part of these key/value pairs is... the s, or b, or i character. That indicates whether the data is String, Binary or Integer).

Well it turns out that taking a straight text password and encrypting it ain't too bad. I'm not going to take credit for this code but the following vb dot net code uses the windows api to do just that.

Imports System
Imports System.Text
Imports System.Runtime.InteropServices
Imports System.ComponentModel
Imports Microsoft.VisualBasic

Public Class DPAPI
<DllImport("Crypt32.dll", SetLastError:=True, CharSet:=System.Runtime.InteropServices.CharSet.Auto)> Private Shared Function CryptProtectData( _
ByRef pPlainText As DATA_BLOB, _
ByVal szDescription As String, _
ByRef pEntropy As DATA_BLOB, _
ByVal pReserved As IntPtr, _
ByRef pPrompt As CRYPTPROTECT_PROMPTSTRUCT, _
ByVal dwFlags As Integer, _
ByRef pCipherText As DATA_BLOB _
) As Boolean
End Function

<DllImport("Crypt32.dll", SetLastError:=True, CharSet:=System.Runtime.InteropServices.CharSet.Auto)> _
Private Shared Function CryptUnprotectData( _
ByRef pCipherText As DATA_BLOB, _
ByRef pszDescription As String, _
ByRef pEntropy As DATA_BLOB, _
ByVal pReserved As IntPtr, _
ByRef pPrompt As CRYPTPROTECT_PROMPTSTRUCT, _
ByVal dwFlags As Integer, _
ByRef pPlainText As DATA_BLOB _
) As Boolean
End Function

<StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Unicode)> _
Friend Structure DATA_BLOB
Public cbData As Integer
Public pbData As IntPtr
End Structure

<StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Unicode)> _
Friend Structure CRYPTPROTECT_PROMPTSTRUCT
Public cbSize As Integer
Public dwPromptFlags As Integer
Public hwndApp As IntPtr
Public szPrompt As String
End Structure

Private Const CRYPTPROTECT_UI_FORBIDDEN As Integer = 1
Private Const CRYPTPROTECT_LOCAL_MACHINE As Integer = 4

Private Shared Sub InitPrompt _
( _
ByRef ps As CRYPTPROTECT_PROMPTSTRUCT _
)
ps.cbSize = Marshal.SizeOf(GetType(CRYPTPROTECT_PROMPTSTRUCT))
ps.dwPromptFlags = 0
ps.hwndApp = IntPtr.Zero
ps.szPrompt = Nothing
End Sub

Private Shared Sub InitBLOB _
( _
ByVal data As Byte(), _
ByRef blob As DATA_BLOB _
)
' Use empty array for null parameter.
If data Is Nothing Then
data = New Byte(0) {}
End If

' Allocate memory for the BLOB data.
blob.pbData = Marshal.AllocHGlobal(data.Length)

' Make sure that memory allocation was successful.
If blob.pbData.Equals(IntPtr.Zero) Then
Throw New Exception( _
"Unable to allocate data buffer for BLOB structure.")
End If

' Specify number of bytes in the BLOB.
blob.cbData = data.Length
Marshal.Copy(data, 0, blob.pbData, data.Length)
End Sub

Public Enum KeyType
UserKey = 1
MachineKey
End Enum

Private Shared defaultKeyType As KeyType = KeyType.UserKey

Public Shared Function Encrypt _
( _
ByVal keyType As KeyType, _
ByVal plainText As String, _
ByVal entropy As String, _
ByVal description As String _
) As String
If plainText Is Nothing Then
plainText = String.Empty
End If
If entropy Is Nothing Then
entropy = String.Empty
End If

Dim result As Byte()
Dim encrypted As String = ""
Dim i As Integer
result = Encrypt(keyType, _
Encoding.Unicode.GetBytes(plainText), _
Encoding.Unicode.GetBytes(entropy), _
description)
For i = 0 To result.Length - 1
encrypted = encrypted & Convert.ToString(result(i), 16).PadLeft(2, "0").ToUpper()
Next
Return encrypted.ToString()
End Function

Public Shared Function Encrypt _
( _
ByVal keyType As KeyType, _
ByVal plainTextBytes As Byte(), _
ByVal entropyBytes As Byte(), _
ByVal description As String _
) As Byte()
If plainTextBytes Is Nothing Then
plainTextBytes = New Byte(0) {}
End If

If entropyBytes Is Nothing Then
entropyBytes = New Byte(0) {}
End If

If description Is Nothing Then
description = String.Empty
End If

Dim plainTextBlob As DATA_BLOB = New DATA_BLOB
Dim cipherTextBlob As DATA_BLOB = New DATA_BLOB
Dim entropyBlob As DATA_BLOB = New DATA_BLOB

Dim prompt As _
CRYPTPROTECT_PROMPTSTRUCT = New CRYPTPROTECT_PROMPTSTRUCT
InitPrompt(prompt)

Try
Try
InitBLOB(plainTextBytes, plainTextBlob)
Catch ex As Exception
Throw New Exception("Cannot initialize plaintext BLOB.", ex)
End Try

Try
InitBLOB(entropyBytes, entropyBlob)
Catch ex As Exception
Throw New Exception("Cannot initialize entropy BLOB.", ex)
End Try

Dim flags As Integer = CRYPTPROTECT_UI_FORBIDDEN

If keyType = keyType.MachineKey Then
flags = flags Or (CRYPTPROTECT_LOCAL_MACHINE)
End If

Dim success As Boolean = CryptProtectData( _
plainTextBlob, _
description, _
entropyBlob, _
IntPtr.Zero, _
prompt, _
flags, _
cipherTextBlob)

If Not success Then
Dim errCode As Integer = Marshal.GetLastWin32Error()

Throw New Exception("CryptProtectData failed.", _
New Win32Exception(errCode))
End If

Dim cipherTextBytes(cipherTextBlob.cbData) As Byte

Marshal.Copy(cipherTextBlob.pbData, cipherTextBytes, 0, _
cipherTextBlob.cbData)

Return cipherTextBytes
Catch ex As Exception
Throw New Exception("DPAPI was unable to encrypt data.", ex)
Finally
If Not (plainTextBlob.pbData.Equals(IntPtr.Zero)) Then
Marshal.FreeHGlobal(plainTextBlob.pbData)
End If

If Not (cipherTextBlob.pbData.Equals(IntPtr.Zero)) Then
Marshal.FreeHGlobal(cipherTextBlob.pbData)
End If

If Not (entropyBlob.pbData.Equals(IntPtr.Zero)) Then
Marshal.FreeHGlobal(entropyBlob.pbData)
End If
End Try
End Function

End Class

Then in our .RDP file generator we just call out to our dot net encryptor like this.

DPAPI.Encrypt(DPAPI.KeyType.MachineKey, "MyPassword", Nothing, "psw")

Now, this is worth stressing again that exposing passwords for RDP clients can be a very dangerous idea. Someone can grab ahold of those and cause all sorts of havoc so your management of these security tools needs to be very well understood. Storage, access, and management of this data is one of the most critical components of your security strategy and anytime your exposing this information you need to be aware of the ramifications of that action. >

Compressing DataSets

chris.bradley — Wed, 11 Feb 2009 18:20:41 GMT

When passing datasets around via webservices we've had some problems with the size of the resulting xml. The following code will compress the dataset and turn it into a byte(). Note that you need to close the streams after calling WriteXml otherwise you'll get a "unexpected end of file" style exception when you try to load back into your dataset.

Dim ms As New IO.MemoryStream
Dim cs As New IO.Compression.GZipStream(ms, IO.Compression.CompressionMode.Compress, False)
results.WriteXml(cs)
cs.Close()
ms.Close()
Dim bytes As Byte() = ms.ToArray

Heres the corresponding decompression code for the server component.

Dim results As New DataSet
Dim ms As New IO.MemoryStream(_Args.Results)
Dim cs As New IO.Compression.GZipStream(ms, IO.Compression.CompressionMode.Decompress, False)
results.ReadXml(cs)
cs.Close()
ms.Close() >>>

Absolutely Awesome!

chris.bradley — Mon, 08 Dec 2008 23:54:24 GMT

We have a huge bank of legacy code written in vb6 that we're slowly porting over to dot net. One of these components that's getting shuffled over this week is our Invoice Printing functionality.

Previously this logic was written in Data Dynamics Active Reports 1.0 (COM based) and we're looking to upgrade this to Active Reports 4.2 (Dot Net 2.0). A quick plug, Active Reports is an absolutely awesome tool for designing and developing business reports. Even their COM-based 1.0 model shows a ton of insight into HOW we should be developing reporting applications.

Anyway, we're going to have to re-write some code as part of this process but the one component that would be nice to pull across is the "design" surface. This basically consists of all the controls and layouts. If we have to re-design this from scratch we're going to miss positions, fonts, etc so it's better to automate this piece if we can. Here's what it looks like...

Now, Active Reports supports a common design layout for AR2.0 to AR4.2, sort of a XML-based layout language they call RPX. If you can save a report layout into an RPX you can load it back later on. Trick is that our reports are AR1.0 and that old version uses a different binary format for saving layouts. I need to somehow upgrade these from AR1.0 to AR2.0 before I can save them into the common format.

Turns out this is easier than you'd think. The vb6 report design files are seperated into two components... a binary .DSX file and a text .DSR file. If you crack open the DSR file the first bit of "code" specifies which tool "manages" the resource. That's what the big ugly guid is. By changing this guid from AR1.0 to AR2.0 we can fake VB6 into thinking it's dealing with an AR2.0 report.

Now, when we open up the designer in vb6, we have the option to save the report layout by using the File, Save feature within the report designer. Likewise in dot net we can use the Report menu item to Load a saved layout. We've managed to save ourselves all the work in positioning, sizing, and configuring all the controls behind this active report!

Now, all this effort to avoid recreating some controls... worth it? To me it is. The problem with me redesiging these reports is that...

1. I'm not that great of a detail guy. Not bad, but not great. I'm positive I'd screw something up. And it may not be a huge deal to the first 20 stores that get this upgrade but at some point that change in field size would be a critical concern to some store. If I can avoid the issue altogether that's a good thing.

2. I get bored easy. I'd copy probably 5 controls before I'd get distracted and switched over to something else. I'll bet you it would take me a week to get this one report copied over. By converting the entire report in one pass I've moved the project to the point where I can have someone else take over the loading of data.

Aristo Instant Messenger

chris.bradley — Wed, 29 Oct 2008 19:28:16 GMT

We just added a cool new feature to our aristo framework, embedded instant messaging! Pretty typical feature but here's what our gui looks like.

Now when we first heard requests for this component I was a little skeptical. It's pretty much a given that at some point ALL software applications get a request to add an instant messenger and email capability; you could be programming a calculator and eventually you'll get this request. Our I.M. does have a couple key differences.

1. This is an internal only application. Staff can't be chatting to their friends and family through this messenger.

2. This is an automatic application. Users don't have to start another app to get the messenger going. It's always on when they're in PBS.

3. This is an embedded component. Because we're running in a Terminal window we have to work whithin our existing UI space (aka the terminal window, no extra desktop apps allowed).

4. Because we're embedded there are some very cool extensions we can add for sending "information" in our software between users. More on this in the next couple weeks.

Here's a quick primer on how this works.

Users now have a messenger icon in their system tray whenever they start Aristo. At login this icon will notify users that they are connected to the aristo messenger (this notification will also point out the fact that there is something new in their software).

Users can right click on this icon to either send a message to a specific user or send a broadcast message. Message windows are pretty much like any other I.M. solution and can be minimized while users are doing actual work. New messages cause the conversation windows to highlight until the user views them.

The more interesting aspect of this component will be HOW we can use it as a tool within our software but we've definitely heard this request a lot and our users should be happy to see we've reacted.

Couple t-sql tricks

chris.bradley — Fri, 29 Aug 2008 03:50:20 GMT

I came across the need to do some t-sql work prior to our annual dealer conference next week. The short version is that we need to sanitize our customer data by replacing customer pictures with some stock photos of people around PBS. I certainly didn't want to have to manually change every customer record so I figured a gigantic t-sql script to update everyone would be a huge help.

Now I gotta admit - this is not the kind of component I normally work on. Database administrators tear through these kinds of apps but when you're a bit more of a generalist you need to be pretty good at googling the right kind of questions.

Anyway, here's the requirements...

1. Update every customer record with a new photo.
2. Choose the photo from a subset of 10 or 12 stock pictures.
3. Try to make sure that "consecutive" customer records don't have the same photo.

The first thing that takes figuring out is how to load an image from a file into a table. For some reason google-ing this didn't do too good. In the end it's pretty easy. The following command loads an image file into a blob-style object for use in t-sql...

SELECT * FROM OPENROWSET(BULK N'c:\creek.jpg', SINGLE_BLOB) as i

Note that you need the rowset descriptor "as i". Otherwise you'll receive some error about corelations.

Next I wanted to build a temporary table of customer id's and picture id's. Something like...

Customer1 Picture1
Customer2 Picture2
Customer3 Picture3
Customer4 Picture1
Customer5 Picture2
...

This would make a somewhat random picture list where at least adjacent customers wouldn't have the same picture. I accomplished this using the ROW_NUMBER() function against our customer code's.

INSERT INTO #Links (ContactId, PictureId)
(SELECT fldId, ROW_NUMBER() OVER(ORDER BY fldCode) % @Image_Count
FROM tblContacts)

Once we've got these two pieces together we can generate our test data pretty reliably. Here's the final sql script ...

DECLARE @Image_Count int    SET @Image_Count = 2

CREATE TABLE #Links (
    ContactId uniqueidentifier,
    PictureId int
)

INSERT INTO #Links (ContactId, PictureId)
    (SELECT fldId, ROW_NUMBER() OVER(ORDER BY fldCode) % @Image_Count
    FROM tblContacts)

UPDATE tblContacts SET fldImage =
    (SELECT * FROM OPENROWSET(BULK N'c:\creek.jpg', SINGLE_BLOB) as i)
WHERE fldId IN (SELECT ContactId FROM #Links WHERE PictureId = 0)

UPDATE tblContacts SET fldImage =
    (SELECT * FROM OPENROWSET(BULK N'c:\dock.jpg', SINGLE_BLOB) as i)
WHERE fldId IN (SELECT ContactId FROM #Links WHERE PictureId = 1)

DROP TABLE #Links

Sourcegear Vault Issue

chris.bradley — Sun, 25 May 2008 04:56:21 GMT

We recently began using the Sourcegear Vault Enhanced Client for Visual Studio. Intermittently we would receive errors accessing our Vault Server. Something to the effect of...

"The Vault server could not be contacted to perform the operation."

You'd try the operation a couple more times and eventually it would go through. After a bunch of digging I found a couple articles on the Sourcegear support forums that pointed me in the direction of the default proxy settings.

http://support.sourcegear.com/viewtopic.php?t=735

What was really interesting was that a> this happened usually when working offsite and b> it happened predominantly through the Visual Studio IDE Client... the normal Vault GUI was fine.

It turns out that this default proxy setting is stored in an app.config. When I compared the VaultGuiClient.exe.config with the devenv.exe.config (Program files\Microsoft VS 2008\Common7\IDE) I found that the Vault GUI had specified to use the system proxy settings while Visual Studio had specified that no proxy existed.

Since changing the devenv.exe.config to match the vault gui config the issue appears to have disappeared. Here's what our new devenv config file looks like... note that the defaultProxy section is the important bit.

The math behind Electronic Document Management

chris.bradley — Mon, 14 Apr 2008 17:41:30 GMT

One of the key ideals behind Electronic Document Management is that it's supposed to save you money. The idea is that by eliminating some of the paper requirements you can cut down on supply and maintenance costs. It's one of those arguments that I've heard a number of times and to be honest I'd mostly ignored it.

My thought was that KNOWING you had electronic copies of each service order was well worth the investment and that the ongoing management and filing of these documents had such a time savings that the financial ramifications weren't as big a deal. Turns out I was wrong...

So here's the premise...

Each work order written inside of a Dealer Management System typically consists of 6 copies.

1 x Hard Copy approving the work.
1 x Technician Copy, so the tech's can note their work.
2 x Closing Copy, one for the customer, one for the dealership.
1 x Warranty Copy for the the manufacturer (if applicable).
1 x Accounting Copy, for the accounting system.

So, a store with 12 technicians doing 5 service orders per day would generate 360 documents per day.

12 techs x 5 service orders x 6 documents per service order = 360 documents

The same store, over the course of a year would generate almost 100,000 service documents.

360 documents per day x 5 days per week x 52 weeks per year = 93,600

Based on our experience each of these documents costs around 4 cents to print. When you take into account paper, toner, and printer wear and tear this adds up pretty quickly.

4 cents per document x 93,600 documents = $3,744 per year.

And these numbers are for an average sized shop!

I still believe that alot of the benefit behind electronic documents is the ability to better manage and store your information, but these kinds of figures do make a compelling financial argument.

Unit Test / Data Generation

chris.bradley — Tue, 26 Feb 2008 22:41:50 GMT

I’ll admit that I have a love/hate relationship with Unit Testing. Unit Testing has absolutely saved my bacon a number of times where a seemingly inconsequential change would have crippled our software. The reality is that the only way you can verify that those tiny changes don’t wreck havoc is to script your testing for known results. By having your build process validate those results you add a very strong safety net to your development process.

But… in most software systems there’s this pesky layer that throws a real wrench into the entire thing – it’s called the database.

The problem is this… a Unit Test is meant to be autonomous. It should be free standing and require no external dependencies. In reality the code you’re executing typically interacts with an existing database and performs INSERTS/UPDATES/DELETES to verify that certain functions still work. What get’s awkward is that the “Unit Test” database is typically an external dependency for your Unit Tests – it just has to be there be in the correct state or the Unit Tests will crash hard. Things like database schema changes, new testing datasets, and changing database locations really screw the unit tests up.

I’m going to describe a solution to this problem that we’ve been using for the last couple weeks. It’s worked extremely well by streamlining issues we had with databast style unit tests and by increasing our code quality. Everything here is based on Visual Studio 2005, nUnit, and nAnt but it’s more the concept that’s interesting.

How It Works!

We started off by adding a special project as part of each module called the Schema. Basically this is a simple class library with a bunch of embedded SQL scripts. These SQL scripts are responsible for taking a database and putting it into a known state for the Unit Tests. Execution of these scripts is done by a class called the SchemaManager, it has three methods which will become obvious in a minute.

The Unit Tests now add a reference to the Schema project. As a nUnit Setup Method we'll call out to a special class called the TestConfiguration. This TestConfiguration will be responsible for triggering the SchemaManager. Here's what a typical Unit Test in our UnitTest project would look like.

The TestConfiguration is a little more interesting. It uses a singleton pattern to create a SchemaManager, tell it to Drop the current Schema, and then Create it fresh. This executes the Drop Table/Procedure commands to clear out our database, and then uses the Create Table commands to make a fresh database.

Perfect! Now the database schema is in a state that our Unit Tests can work with – table are there, stored procedures are there, life’s good.

Now we need to create some canned data before we start our tests. Things like maintenance objects, customer codes, etc. are all expected to be in the database before our Unit Tests run so we need to create scripts in our UnitTest database which setup this data. Something like this...

We’ll embed this scripts into the UnitTest project and then change our TestConfiguration so that it gets the SchemaManager to insert this data after calling CreateSchema. We'll simply use the ExecuteResourceSql call we defined above and call it from our TestConfiguration.

And that's pretty much it! I think the beauty of this SchemaManager is that is makes the database a source-controlled resource. Instead of having some unknown dependency that the unit tests must rely on, the unit tests actually create their environment using scripts generated by the developer. If the developer breaks the Schema or the Business Logic, the unit test will catch it right away and any failed unit tests are expected to be logic errors instead of logistical problems between the development and unit test environments.

If you have any comments or suggestions on this area please drop me a line - it's a very interesting (and often underused) development practice!

PS - The foundation for a lot of this stuff is from a 4guysfromrolla posting, check it out for more details...

http://aspnet.4guysfromrolla.com/articles/040605-1.2.aspx

The Danger of Little Issues...

chris.bradley — Sat, 09 Feb 2008 07:37:44 GMT

I came across a great article a couple weeks back about the danger of little issues... the foundation of this article was that every day we get interupted in ways that force us to change what we're focused on, deal with an emergency, and then re-focus back on the task at hand.

This articule suggested that these "shifts" cause us to lose more time then if we could just merrily work through one project at a time... makes sense, huh?

What's really interesting is that this article suggested that the LITTLE things actually cause you more long-term grief than the big things. The reasoning was that when you come across a big issue you learn in a big way... and subconsciously take action to resolve it in the future. The problem with little issues is that they only take 1/2 an hour to resolve... and you become complacent in resolving them... twice a week... for a year.

Why is this worth mentioning? Today while setting up for our NADA trade show (www.nada.org) I ended up spending an hour trouble-shooting some file location issues on our demo server. It's a stupid virtual-drive issue that only occurs in certain environments and generally takes me about an hour to figure out.

The trouble is that we've grown complacent in spending an hour on this issue... instead of focusing on why we need a virtual-drive in the first place. If we'd reworked our pathing a year ago, I wouldn't get bothered by this every few weeks.

Once you identify the "Little Issue" it's all about committing yourself to resolving it... we're in mad-panic trade show mode right now so let's see what we can do after we get back from San Francisco...

To be continued...

Cruise Control .Net / Queued Building

chris.bradley — Tue, 08 Jan 2008 01:40:02 GMT

I just found an outstanding configuration element inside of cruise control which allows you to setup multiple projects to run in a queued environment. Basically it functions as a mutex so that while one project is building all other projects go into a queue until the first project is completed, and then another one is given control. The configuration element is really easy, here's our ccnet.config...

<project name="Source Control" queue="ReleaseBuild">
    <workingDirectory>C:\Projects\dvt-root</workingDirectory>
    <triggers>
      <intervalTrigger seconds="180" />
    </triggers>
    <modificationDelaySeconds>10</modificationDelaySeconds>
    <sourcecontrol type="vault" autoGetSource="true" applyLabel="false">
      <executable>c:\program files\sourcegear\vault client\vault.exe</executable>
      <username>BuildUserName</username>
      <password>BuildPassword</password>
      <host>myDevtServer</host>
      <repository>Evolution</repository>
      <folder>$/Dvt-Root</folder>
      <useWorkingDirectory>false</useWorkingDirectory>
      <ssl>False</ssl>
      <setFileTime>current</setFileTime>
    </sourcecontrol>
</project>

<project name="Release Esenaar" queue="ReleaseBuild">
    <workingDirectory>C:\Projects\dvt-root</workingDirectory>
    <triggers>
      <intervalTrigger seconds="180" />
    </triggers>
    <tasks>
       <nant>
        <baseDirectory></baseDirectory>
        <buildArgs>-D:debug=False</buildArgs>
        <buildFile>Release.Esenaar.Build</buildFile>
        <buildTimeoutSeconds>1200</buildTimeoutSeconds>
        <targetList>
          <target>build</target>
        </targetList>
      </nant>
    </tasks>
</project>

<project name="Release Duvel" queue="ReleaseBuild">
    <workingDirectory>C:\Projects\dvt-root</workingDirectory>
    <triggers>
      <intervalTrigger seconds="180" />
    </triggers>
    <tasks>
       <nant>
        <baseDirectory></baseDirectory>
        <buildArgs>-D:debug=False</buildArgs>
        <buildFile>Release.Duvel.Build</buildFile>
        <buildTimeoutSeconds>1200</buildTimeoutSeconds>
        <targetList>
          <target>build</target>
        </targetList>
      </nant>
    </tasks>
</project>

</cruisecontrol>

The first "project", called "Source Control" is actually responsible for downloading all our code from SourceGear Vault. You can see that every 180 seconds it downloads the source code from vault to a local folder called C:\Projects\dvt-root.

The second and third projects, called "Duvel" and "Esenaar" (maintenance release names), actually call out to our nant script and compile the appropriate release.

Now, our primary goal with all this is to make sure that we don't pull down source code while we're compiling a release (and vice versa) and that's where the queue property on the project comes into play. By setting the queue on all three projects to be the same thing, cruise control will not allow two projects to execute at the same time!

http://confluence.public.thoughtworks.org/display/CCNET/Project+Configuration+Block

SigPlusNet Missing Module Exception

chris.bradley — Mon, 31 Dec 2007 17:13:52 GMT

As part of our ongoing eSignature rollout we found that a smattering of client machines have issues when attempting to load the SigPlusNet.dll.

This dll is basically our interface to the signature pad itself and would fail with a FileNotFoundException. What was really strange was that the exception contained no information about the file it was trying to load. In addition, 95% of clients worked fine the first time. On the remaining 5% as soon as we started troubleshooting by installing Windows Updates or any other sample apps the error would simply go away.

After using a great little tool called FileMon (www.sysinternals.com) I found that the SigPlusNet.dll was failing on an attempt to find and load msvcr71.dll. Apparently msvcr71.dll is basically a set of C style application methods that are somehow used internally by SigPlusNet. Since this dll is pretty common it appears that on most machines it's installed by something else and so that's why on 95% of machines we had no issues.

Once I modified the install to include this dll, life is wonderful. A very strange unexpected dependancy...

Drivers License Scanner

chris.bradley — Fri, 09 Nov 2007 16:56:12 GMT

In our most recent release of Aristo Contact Management we've added support for a USB Drivers License Scanner. The basis of this project is that we connect this scanner to the local pc, integrate it into our customer screen, and then use the scanner to read customer information.

We've tried to make this a very simple process... after the user selects the Scan button the reader will whir and buzz as it attempts to pull information from the license. The reader itself accomplishes this by taking a picture of the license and then using text recognition (OCR) to separate the license into different fields we can populate our customer record with.

Once we read this information it automatically updates into the appropriate fields on our screen. In addition, because we're taking a picture of the license, we can also attempt to save a picture of the customer IF the regional license supports it.

At this point we support Canadian and US licenses and pull Name, Address, License and expiry fields.

Now, as a promotion for the first 5 clients who email me (and use Aristo Accounting and Contact Management) we're going to send you a drivers license scanner free! Quantities are limited so drop me an email at freeScanner@pbssystems.com and we'll contact the winners.

Enhanced eSignature Changes

chris.bradley — Tue, 18 Sep 2007 17:12:56 GMT

With eSignature rolling into larger beta it was time to address some flexibility concerns that had come up at our second pilot store.

Their chief problem was that when they took a signature on a Hard Copy they wanted the software to automatically print two copies at two different printers - one at the Service Counter (for the customer to take home) and one at the back counter (for the technicians). With our first version of eSignature this was awkward because the user was given one prompting screen and would need to use the "print" dialog twice - pretty difficult in a fast paced retail environment.

So... while we were out at our annual dealer conference last week I bounced this idea around with a couple service managers. We came up with the idea of "Document Actions". The idea is that eSignature already managed the layout of the document, it's not that big a stretch to allow it to also manage WHAT we do with the document once we have it. If we allow our document management infrastructure to take care of this we can specify that this document should be printed on Printer A, then printed on Printer B, and finally reviewed with Printer C. The flexibility can all be managed in our Document Layout interface and tailored to the dealerships processes.

Finally it's worth noting that all this new Document Management is only kicked off if you "Enable Document Management" for that specific document type. Once that checkbox is enabled the background, signature capture, and printing is all handled by eSignature - the existing printing control in Service is totally ignored.

Dot Net / COM Interop Issues

chris.bradley — Wed, 15 Aug 2007 02:19:33 GMT

Just overcome an extremely challenging problem we've been having. Our COM application was calling out to a dot net assembly through COM Interop and when the dot net assembly returned a response the process would get terminated... no friendly output, error messages, or log entries - just dead.

The solution was to totally REMOVE the Dot Net 2.0 framework and re-install. We tried a REPAIR first but that didn't seem to do anything.

It's worth mentioning that the problem didn't occur when calling into the dot net assembly but rather when it returned something to VB6... very strange.

Anyway, strike another issue off the project wall. Time to move back on to Prospecting!

Addendum> Apparently this issue is caused by one of the Dot Net 2.0 Security Updates from Windows Update! For now we're avoiding this patch.

Aristo Vehicle Management (Alpha)

chris.bradley — Fri, 10 Aug 2007 22:53:18 GMT

Yesterday we installed Aristo Vehicle Managment into Alpha at a dealership here in Calgary. We'll be working with them over the next few weeks to iron out any wrinkles as we continue to push forward on our Aristo Front End project. Couple highlights...

A true common vehicle file (which provides for some extremely powerful and concise search capabilities)...

The tree in the top-right corner allows searching by vehicle type according to Make, Model, and Trim level. This search is hierarchical so you can search for any type of unit. We also support search for specific unit types, options, pricing, or even across vehicles that have been sold to customers for potential lease returns.

We've also completed an integrated vehicle screen accessible from all modules. This screen allows all departments to have the same view of a vehicle complete with all vehicle history.

This common UI means that given the right security a user can review and edit any information of a vehicle by selecting the correct View in the top right hand corner. Vehicle history is reported down the right side of the screen and allows immediate access to any "event" (such as a Repair Order or Sales Rep Hold) which has affected this unit.

This project will be the foundation for our further work on Front End and provides a major step forward on the Aristo Front End project.

Software Development Plan

chris.bradley — Tue, 24 Jul 2007 04:28:20 GMT

Our Front End Aristo project is going to be managed by what we call a Product Team. A Product Team is basically three individuals who will make it their top priority to achieve deployment of the Aristo Front End application. Each of these individuals has an inherent goal for the project and by working together as a team will come to a consensus on these goals. The Team is made up of the following...

Strategic Product Manager (Dave)

The Strategic Product Manager is responsible for understanding what the market demands of the product. It is their responsibility to understand business cases, market requirements, as well as scheduling demands.

Technical Product Manager (Chris)

The Technical Product Manager works to understand what can be built in what amount of time to satisfy the market demands. Their responsibility is to help estimate project timelines and to ensure project milestones are accomplished.

Sales Product Manager (Pat C)

The Sales Product Manager focuses on understanding how the product will be received by the market. Their responsibility is to be THE product expert, to act as a sales and installation resource, and to create excitement around the product.

Starting asap we're going to convene the Product Team daily for 1 hour - no longer - to discuss the finer requirements for this project. Based on these meetings we will create a project schedule defining which features will be implemented in which iteration. Our schedule will be have five 1-week iterations with our Front End module ready for install by the end of our final iteration.

It's an exciting time for this project; by having three resources each focusing on their specific "needs" for this project we should have faster completion, more complete requirements, and a more exciting product release.

Musings on Framework

chris.bradley — Tue, 08 May 2007 09:53:14 GMT

Over the past 6 months we've seen a certain emergence of application knowledge in our framework that I'm not entirely sure I understand. Assemblies such as SOA and TitaniumDataAccess provide extremely usefull functionality but are deeply rooted in a number of applications which makes you wonder on how much they really should be in framework... after all Framework is about more than adding an assembly reference and having intellisense.

One area that has shown promise is the factory-style interface structure used in eSignature. We build our... ahem... ServiceContract assembly - which has no Framework references - inside the module, then copy the pre-compiled assembly to our consumer module, hard-referencing it. At runtime we create a module instruction which says, register this interface ICustomer with this class.

There are a couple of benefits to this style of communication. First, our current message based soa is pretty procedural, an interface based soa is more Object-Oriented which should make cross-module communication easier to use; think about the "conversationId" parameters we've had to add to ContactServices. Secondly, because this feels more Object-Oriented, it's easier to explain this type of communication to developers. Finally, it's easier to create mock handlers for our unit tests.

The only down-side is how do you introduce something like this without destroying our current communication scheme and blowing away advent integration. The only real feasible way is to attempt to use it in a "limited" mode with one module, maybe across the sales module in our next service pack.

One other idea would be to have the modules define their own resource strings and images with a similar "initialization" instruction. That way your globalization dll could change very little and if an invalid resource is requested it could return a "no smoking sign" and not break the module.

Couple big changes but worth looking at for future development.