# Monday, April 13, 2009

Panama Canal Cruise Days 1-5

Five days after arriving on our cruise ship, I've just discovered how easy it is to get on the internet. Bad news for Andrea...

We boarded in Fort Lauderdale, and yes, average age on our cruise is around 85. None-the-less, we've been having a great time.

Our first port was half moon cay, a bahamian island currently being leased by Holland America. It's still a little farther north so the water wasn't extremely warm but I can vouch for the fact that if you forget to put suntan lotion on your legs, you'll still burn.

Next stop was Aruba where we went Scuba diving. We dove on an old refining ship and had a run in with an Eel. Later on our dive master informed us he was once bitten by one and required 28 stiches.

Yesertday we went to Curacao, another less touristy, Dutch island. We did a tour of the more remote eastern end of the island and got some good shots of the very colorful capital - Willamstad.

We're at sea now and we should hit the Panama canal early tommorow morning.

#    Comments [0] |
# Monday, March 23, 2009

Source Code Version Control

One area of concern that I come up against quite often is the issue of source code version control. At its most basic layer this is nothing more than being able to control what a version of your software consists of... version A versus version B, that sort of thing. This model works great and is well documented on smaller isolated applications but tends to blur a little when working with larger systems. Specifically we’ve had issues managing system dependencies, as well as change requirements on a larger application consisting of about roughly 50 assemblies. I’ll spend just a few minutes here putting pen to paper on how we’ve structured our development tree to accommodate this.

We’ll start at a pretty simple level; each major release is placed in a named folder based on the version name. Our first branch within that folder is named Major.Minor.Zero, Major and Minor are related to the version name while the Zero indicated this is the opening development branch. Here’s how that looks when we start a new major release... our friendly version name is Jupiler and the Major.Minor associated with this is 5.1.

Notice that within the 5.1.0 folder we have a number of sub-folders. These are what we call modules and basically relate to a VS.Net solution; each one consists of roughly 5-6 projects and can be thought of as a sub-system. These sub-systems collaborate to provide our applications functionality.

Now, when we release our major version, Jupiler, we’ll typically LOCK the 5.1.0 folder so that no further changes can be made. This code is set in stone and we want to force all new changes to be done in an isolated environment until they’re tested and ready to go. Invariable though, something critical comes up during rollout and we need to make code changes to address those concerns. When this situation arises we BRANCH into a new folder called Major.Minor.One. This code folder consists of all the new development changes required to add what we call a Service Pack to the release. Here’s how this looks.  

One interesting aspect of this layout is that we only branch the modules that require changes. These Service Pack’s generally don’t require a lot of wholesale change; they`re usually small isolated bugs/features which can be addressed in a single module. You can see in the above diagram that our 5.1.1 only has changes for Accounting, all the other modules are identical to how they were developed in 5.1.0. This gives us a few key benefits.

1.       It limits the amount of open source code.

2.       It makes it easier to identify what code is actively being changed for a service pack.

3.       It leaves the original branch open so that it can be reviewed/modified for OTHER unanticipated issues.

Final rollout of the new changes is done by releasing 5.1.1 of Accounting with all the previous versions of the other modules.

#    Comments [0] |
# Wednesday, March 11, 2009

Congratulations, you've installed dasBlog with Web Deploy!

After logging in, be sure to visit all the options under Configuration in the Admin Menu Bar above. There are 26 themes to choose from, and you can also create your own.

 

#    Comments [0] |
# Friday, March 06, 2009

MSTSC Username/Password

Not that this is necessarily a good idea. But after way too much time googling this, I think I'll clear up the fact that it IS possible call the Microsoft RDP client with a username and password parameter.

Now, everyone out there will say, yeah - we know that. You key in the username and password and if you check off the "remember credentials" checkbox it will store that data.

Well the trick is that this data is stored in a place that's not easily accessed -Windows Stored UserNames and Passwords. This area is basically a secured area of windows where passwords are stored for remote connections. You can access this by going Control Panel\User Accounts\Manager your network passwords. Here's a screen snap.

Our management infrastructure tracks this username and password in a seperate secured database, and then when we call MSTSC.exe from our software we want to pass this information. Now, a little more information on how you can pass parameters to MSTSC is in order. When you call MSTSC you can pass as a parameter a .RDP file which contains information about how you want to connect to the remote computer. Same idea as every other document type in Windows, a .RDP file is associated with MSTSC and is used to contain information for that application. Fortunately the .RDP files are basically key value pairs. Check it out...

screen mode id:i:1
desktopwidth:i:1024
desktopheight:i:768
session bpp:i:16
winposstr:s:0,1,526,61,1566,865
full address:s:SomeIpAddress
compression:i:1
keyboardhook:i:2
audiomode:i:0
redirectdrives:i:0
redirectprinters:i:1
redirectcomports:i:0
redirectsmartcards:i:1
displayconnectionbar:i:0
autoreconnection enabled:i:1
alternate shell:s:
shell working directory:s:
disable wallpaper:i:0
disable full window drag:i:1
disable menu anims:i:1
disable themes:i:0
disable cursor setting:i:0
bitmapcachepersistenable:i:1
allow desktop composition:i:1
allow font smoothing:i:0
redirectclipboard:i:1
redirectposdevices:i:0
authentication level:i:2
prompt for credentials:i:0
negotiate security layer:i:1
remoteapplicationmode:i:0
gatewayhostname:s:
gatewayusagemethod:i:4
gatewaycredentialssource:i:4
gatewayprofileusagemethod:i:0
promptcredentialonce:i:1
drivestoredirect:s:

The imporant pieces here is the "full address" line... this points to the server we're connecting to.

Well... it turns out you can also pass parameters for domain, username, and password. You just need to add them to the bottom of your .rdp file like this

domain:s:MyDomain
username:s:MyUserName
password 51:b:MyPassword

But wait a second, saving a password in plain text in a .rdp file sounds like just about the worst security hole you could use! And you're right - you can't actually pass the password parameter as straight text. It needs to be encrypted binary.

(As an aside, that's what the second part of these key/value pairs is... the s, or b, or i character. That indicates whether the data is String, Binary or Integer).

Well it turns out that taking a straight text password and encrypting it ain't too bad. I'm not going to take credit for this code but the following vb dot net code uses the windows api to do just that.

Imports System
Imports System.Text
Imports System.Runtime.InteropServices
Imports System.ComponentModel
Imports Microsoft.VisualBasic

Public Class DPAPI
<DllImport("Crypt32.dll", SetLastError:=True, CharSet:=System.Runtime.InteropServices.CharSet.Auto)> Private Shared Function CryptProtectData( _
ByRef pPlainText As DATA_BLOB, _
ByVal szDescription As String, _
ByRef pEntropy As DATA_BLOB, _
ByVal pReserved As IntPtr, _
ByRef pPrompt As CRYPTPROTECT_PROMPTSTRUCT, _
ByVal dwFlags As Integer, _
ByRef pCipherText As DATA_BLOB _
) As Boolean
End Function

<DllImport("Crypt32.dll", SetLastError:=True, CharSet:=System.Runtime.InteropServices.CharSet.Auto)> _
Private Shared Function CryptUnprotectData( _
ByRef pCipherText As DATA_BLOB, _
ByRef pszDescription As String, _
ByRef pEntropy As DATA_BLOB, _
ByVal pReserved As IntPtr, _
ByRef pPrompt As CRYPTPROTECT_PROMPTSTRUCT, _
ByVal dwFlags As Integer, _
ByRef pPlainText As DATA_BLOB _
) As Boolean
End Function

<StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Unicode)> _
Friend Structure DATA_BLOB
Public cbData As Integer
Public pbData As IntPtr
End Structure

<StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Unicode)> _
Friend Structure CRYPTPROTECT_PROMPTSTRUCT
Public cbSize As Integer
Public dwPromptFlags As Integer
Public hwndApp As IntPtr
Public szPrompt As String
End Structure

Private Const CRYPTPROTECT_UI_FORBIDDEN As Integer = 1
Private Const CRYPTPROTECT_LOCAL_MACHINE As Integer = 4

Private Shared Sub InitPrompt _
( _
ByRef ps As CRYPTPROTECT_PROMPTSTRUCT _
)
ps.cbSize = Marshal.SizeOf(GetType(CRYPTPROTECT_PROMPTSTRUCT))
ps.dwPromptFlags = 0
ps.hwndApp = IntPtr.Zero
ps.szPrompt = Nothing
End Sub

Private Shared Sub InitBLOB _
( _
ByVal data As Byte(), _
ByRef blob As DATA_BLOB _
)
' Use empty array for null parameter.
If data Is Nothing Then
data = New Byte(0) {}
End If

' Allocate memory for the BLOB data.
blob.pbData = Marshal.AllocHGlobal(data.Length)

' Make sure that memory allocation was successful.
If blob.pbData.Equals(IntPtr.Zero) Then
Throw New Exception( _
"Unable to allocate data buffer for BLOB structure.")
End If

' Specify number of bytes in the BLOB.
blob.cbData = data.Length
Marshal.Copy(data, 0, blob.pbData, data.Length)
End Sub

Public Enum KeyType
UserKey = 1
MachineKey
End Enum

Private Shared defaultKeyType As KeyType = KeyType.UserKey

Public Shared Function Encrypt _
( _
ByVal keyType As KeyType, _
ByVal plainText As String, _
ByVal entropy As String, _
ByVal description As String _
) As String
If plainText Is Nothing Then
plainText = String.Empty
End If
If entropy Is Nothing Then
entropy = String.Empty
End If

Dim result As Byte()
Dim encrypted As String = ""
Dim i As Integer
result = Encrypt(keyType, _
Encoding.Unicode.GetBytes(plainText), _
Encoding.Unicode.GetBytes(entropy), _
description)
For i = 0 To result.Length - 1
encrypted = encrypted & Convert.ToString(result(i), 16).PadLeft(2, "0").ToUpper()
Next
Return encrypted.ToString()
End Function

Public Shared Function Encrypt _
( _
ByVal keyType As KeyType, _
ByVal plainTextBytes As Byte(), _
ByVal entropyBytes As Byte(), _
ByVal description As String _
) As Byte()
If plainTextBytes Is Nothing Then
plainTextBytes = New Byte(0) {}
End If

If entropyBytes Is Nothing Then
entropyBytes = New Byte(0) {}
End If

If description Is Nothing Then
description = String.Empty
End If

Dim plainTextBlob As DATA_BLOB = New DATA_BLOB
Dim cipherTextBlob As DATA_BLOB = New DATA_BLOB
Dim entropyBlob As DATA_BLOB = New DATA_BLOB

Dim prompt As _
CRYPTPROTECT_PROMPTSTRUCT = New CRYPTPROTECT_PROMPTSTRUCT
InitPrompt(prompt)

Try
Try
InitBLOB(plainTextBytes, plainTextBlob)
Catch ex As Exception
Throw New Exception("Cannot initialize plaintext BLOB.", ex)
End Try

Try
InitBLOB(entropyBytes, entropyBlob)
Catch ex As Exception
Throw New Exception("Cannot initialize entropy BLOB.", ex)
End Try

Dim flags As Integer = CRYPTPROTECT_UI_FORBIDDEN

If keyType = keyType.MachineKey Then
flags = flags Or (CRYPTPROTECT_LOCAL_MACHINE)
End If

Dim success As Boolean = CryptProtectData( _
plainTextBlob, _
description, _
entropyBlob, _
IntPtr.Zero, _
prompt, _
flags, _
cipherTextBlob)

If Not success Then
Dim errCode As Integer = Marshal.GetLastWin32Error()

Throw New Exception("CryptProtectData failed.", _
New Win32Exception(errCode))
End If

Dim cipherTextBytes(cipherTextBlob.cbData) As Byte

Marshal.Copy(cipherTextBlob.pbData, cipherTextBytes, 0, _
cipherTextBlob.cbData)

Return cipherTextBytes
Catch ex As Exception
Throw New Exception("DPAPI was unable to encrypt data.", ex)
Finally
If Not (plainTextBlob.pbData.Equals(IntPtr.Zero)) Then
Marshal.FreeHGlobal(plainTextBlob.pbData)
End If

If Not (cipherTextBlob.pbData.Equals(IntPtr.Zero)) Then
Marshal.FreeHGlobal(cipherTextBlob.pbData)
End If

If Not (entropyBlob.pbData.Equals(IntPtr.Zero)) Then
Marshal.FreeHGlobal(entropyBlob.pbData)
End If
End Try
End Function

End Class


Then in our .RDP file generator we just call out to our dot net encryptor like this.

DPAPI.Encrypt(DPAPI.KeyType.MachineKey, "MyPassword", Nothing, "psw")

Now, this is worth stressing again that exposing passwords for RDP clients can be a very dangerous idea. Someone can grab ahold of those and cause all sorts of havoc so your management of these security tools needs to be very well understood. Storage, access, and management of this data is one of the most critical components of your security strategy and anytime your exposing this information you need to be aware of the ramifications of that action.

#    Comments [1] |
# Wednesday, February 11, 2009

Compressing DataSets

When passing datasets around via webservices we've had some problems with the size of the resulting xml. The following code will compress the dataset and turn it into a byte(). Note that you need to close the streams after calling WriteXml otherwise you'll get a "unexpected end of file" style exception when you try to load back into your dataset.

Dim ms As New IO.MemoryStream
Dim cs As New IO.Compression.GZipStream(ms, IO.Compression.CompressionMode.Compress, False)
results.WriteXml(cs)
cs.Close()
ms.Close()
Dim bytes As Byte() = ms.ToArray

Heres the corresponding decompression code for the server component.

Dim results As New DataSet
Dim ms As New IO.MemoryStream(_Args.Results)
Dim cs As New IO.Compression.GZipStream(ms, IO.Compression.CompressionMode.Decompress, False)
results.ReadXml(cs)
cs.Close()
ms.Close()

#    Comments [0] |
# Monday, January 19, 2009

Creative Mind @ Work!

It's just so real... you think you could reach out and touch it!

#    Comments [1] |
# Monday, December 08, 2008

Absolutely Awesome!

We have a huge bank of legacy code written in vb6 that we're slowly porting over to dot net. One of these components that's getting shuffled over this week is our Invoice Printing functionality.

Previously this logic was written in Data Dynamics Active Reports 1.0 (COM based) and we're looking to upgrade this to Active Reports 4.2 (Dot Net 2.0). A quick plug, Active Reports is an absolutely awesome tool for designing and developing business reports. Even their COM-based 1.0 model shows a ton of insight into HOW we should be developing reporting applications.

Anyway, we're going to have to re-write some code as part of this process but the one component that would be nice to pull across is the "design" surface. This basically consists of all the controls and layouts. If we have to re-design this from scratch we're going to miss positions, fonts, etc so it's better to automate this piece if we can. Here's what it looks like...

Now, Active Reports supports a common design layout for AR2.0 to AR4.2, sort of a XML-based layout language they call RPX. If you can save a report layout into an RPX you can load it back later on. Trick is that our reports are AR1.0 and that old version uses a different binary format for saving layouts. I need to somehow upgrade these from AR1.0 to AR2.0 before I can save them into the common format.

Turns out this is easier than you'd think. The vb6 report design files are seperated into two components... a binary .DSX file and a text .DSR file. If you crack open the DSR file the first bit of "code" specifies which tool "manages" the resource. That's what the big ugly guid is. By changing this guid from AR1.0 to AR2.0 we can fake VB6 into thinking it's dealing with an AR2.0 report.

   

Now, when we open up the designer in vb6, we have the option to save the report layout by using the File, Save feature within the report designer. Likewise in dot net we can use the Report menu item to Load a saved layout. We've managed to save ourselves all the work in positioning, sizing, and configuring all the controls behind this active report!

 

Now, all this effort to avoid recreating some controls... worth it? To me it is. The problem with me redesiging these reports is that...

1. I'm not that great of a detail guy. Not bad, but not great. I'm positive I'd screw something up. And it may not be a huge deal to the first 20 stores that get this upgrade but at some point that change in field size would be a critical concern to some store. If I can avoid the issue altogether that's a good thing.

2. I get bored easy. I'd copy probably 5 controls before I'd get distracted and switched over to something else. I'll bet you it would take me a week to get this one report copied over. By converting the entire report in one pass I've moved the project to the point where I can have someone else take over the loading of data.

#    Comments [0] |
# Tuesday, November 04, 2008

SQL Server Collations

For some strange reason, our software requires SQL Server to run with a specific collation. Basically the collation specifies the "order" in which text data is stored and has an impact on how indexes manage and search through large amounts of data.

Our issue comes up when the DATABASE collation doesn't match the SERVER collation. We'll get something like this...

SqlException cannot resolve the collation conflict between 'Latin1_General_CP1_CI_AS' and 'SQL_Latin1_General_CP1_CI_AS'.

All I can tell you from this exception is that the two collations don't match and our software won't run. Essentially we have to re-install sql server to reset the collation on the server so that it matches the database.

You'd think this would be a pretty easy configuration option during the sql server install... not so. During the sql install you do get a prompt to choose the server collation but the list is language based... nothing mentioning SQL_Lation1_SomethingOrRather.

There is an option on there for legacy collations, since our app was developed on an older version of SQL Server this would make sense, but again nothing mentioning SQL_Latin1.

Well, turns out that the option for "Dictionary order, case-insensitive, for use with 1252 Character set" is actually equivelant to our SQL_Latin1 collation and once the database server comes up sure enough it's been configured properly.

Tricky config setting but at least now we'll know what to look for on new installs. I'd still love to know WHY we need to run with that specific setting...

 

#    Comments [2] |
# Wednesday, October 29, 2008

Aristo Instant Messenger

We just added a cool new feature to our aristo framework, embedded instant messaging! Pretty typical feature but here's what our gui looks like.

 

Now when we first heard requests for this component I was a little skeptical. It's pretty much a given that at some point ALL software applications get a request to add an instant messenger and email capability; you could be programming a calculator and eventually you'll get this request. Our I.M. does have a couple key differences.

   1. This is an internal only application. Staff can't be chatting to their friends and family through this messenger.

   2. This is an automatic application. Users don't have to start another app to get the messenger going. It's always on when they're in PBS.

   3. This is an embedded component.  Because we're running in a Terminal window we have to work whithin our existing UI space (aka the terminal window, no extra desktop apps allowed).

   4. Because we're embedded there are some very cool extensions we can add for sending "information" in our software between users. More on this in the next couple weeks.

 

Here's a quick primer on how this works.

Users now have a messenger icon in their system tray whenever they start Aristo. At login this icon will notify users that they are connected to the aristo messenger (this notification will also point out the fact that there is something new in their software).

Users can right click on this icon to either send a message to a specific user or send a broadcast message. Message windows are pretty much like any other I.M. solution and can be minimized while users are doing actual work. New messages cause the conversation windows to highlight until the user views them.

The more interesting aspect of this component will be HOW we can use it as a tool within our software but we've definitely heard this request a lot and our users should be happy to see we've reacted.

#    Comments [0] |
# Thursday, August 28, 2008

Couple t-sql tricks

I came across the need to do some t-sql work prior to our annual dealer conference next week. The short version is that we need to sanitize our customer data by replacing customer pictures with some stock photos of people around PBS. I certainly didn't want to have to manually change every customer record so I figured a gigantic t-sql script to update everyone would be a huge help.

Now I gotta admit - this is not the kind of component I normally work on.  Database administrators tear through these kinds of apps but when you're a bit more of a generalist you need to be pretty good at googling the right kind of questions.

Anyway, here's the requirements...

1. Update every customer record with a new photo.
2. Choose the photo from a subset of 10 or 12 stock pictures.
3. Try to make sure that "consecutive" customer records don't have the same photo.

The first thing that takes figuring out is how to load an image from a file into a table. For some reason google-ing this didn't do too good. In the end it's pretty easy. The following command loads an image file into a blob-style object for use in t-sql...

SELECT * FROM OPENROWSET(BULK N'c:\creek.jpg', SINGLE_BLOB) as i

Note that you need the rowset descriptor "as i". Otherwise you'll receive some error about corelations.

Next I wanted to build a temporary table of customer id's and picture id's. Something like...

Customer1 Picture1
Customer2 Picture2
Customer3 Picture3
Customer4 Picture1
Customer5 Picture2
...

This would make a somewhat random picture list where at least adjacent customers wouldn't have the same picture. I accomplished this using the ROW_NUMBER() function against our customer code's.

INSERT INTO #Links (ContactId, PictureId)
    (SELECT fldId, ROW_NUMBER() OVER(ORDER BY fldCode) % @Image_Count
    FROM tblContacts)

Once we've got these two pieces together we can generate our test data pretty reliably. Here's the final sql script ...

DECLARE @Image_Count int    SET @Image_Count = 2

CREATE TABLE #Links (
    ContactId uniqueidentifier,
    PictureId int
)

INSERT INTO #Links (ContactId, PictureId)
    (SELECT fldId, ROW_NUMBER() OVER(ORDER BY fldCode) % @Image_Count
    FROM tblContacts)



UPDATE tblContacts SET fldImage =
    (SELECT * FROM OPENROWSET(BULK N'c:\creek.jpg', SINGLE_BLOB) as i)
WHERE fldId IN (SELECT ContactId FROM #Links WHERE PictureId = 0)

UPDATE tblContacts SET fldImage =
    (SELECT * FROM OPENROWSET(BULK N'c:\dock.jpg', SINGLE_BLOB) as i)
WHERE fldId IN (SELECT ContactId FROM #Links WHERE PictureId = 1)



DROP TABLE #Links

#    Comments [0] |