Friday, January 29, 2016

Mocking the Database

I believe I mentioned before that I worked with a custom, home-grown ORM at one of my clients.  If I didn't mention that, I'm mentioning it now.  One of the biggest problems I had with the solution we implemented was that we didn't have any tests for it.  Every now and then we'd encounter a new scenario (like returning a list of objects instead of a single object) and we'd have to code it in, with no way to make sure we weren't breaking what was already there.

One (particularly slow) day I decided enough was enough and I set out to create automated unit tests for every method in that behemoth.  That's when I encountered the problem: how do you mock a database for consumption by ADO.NET?  It turns out it's pretty easy and pretty straightforward.  First off, I found this code somewhere else (which, I know, technically violates the name of the blog, but it is what it is).  You can check out the original post here if you're interested.  I had to make a few changes for my version of MOQ and my specific circumstances, but that's the blog that got me started down the right path.

Here's the high level overview of the ORM: go get data using a DbDataReader, read through the columns in the reader, and for each column, find a corresponding property on the object and map the value from the reader to the property.  It's honestly pretty straightforward.  All I had to figure out was how to mock a DbDataReader.  Here it is:


   1:  private Mock<DbDataReader> CreateFakeDbDataReader(int numberOfFields, int numberOfReads = 1,
   2:              MockBehavior mockBehavior = MockBehavior.Loose, bool hasRows = true)
   3:  {
   4:      // create a mock repository (database)
   5:      var repository = new MockRepository(mockBehavior);
   6:      // create a reader for the mocked repository
   7:      var moqReader = repository.Create<DbDataReader>();
   8:      // setup the reader so that it always has rows (or indicates that it has rows anyway)
   9:      moqReader.SetupGet(p => p.HasRows).Returns(hasRows);
  10:      // setup the reader to indicate it has the specified number of fields
  11:      moqReader.SetupGet(p => p.FieldCount).Returns(numberOfFields);
  12:   
  13:      // readCounter is used to allow the reader to be iterated
  14:      // incrementing readCounter in the callback allows the reader to be read a specific number of times
  15:      var readCounter = 0;
  16:      moqReader.Setup(x => x.Read()).Returns(() => readCounter < numberOfReads).Callback(() => readCounter++);
  17:   
  18:      return moqReader;
  19:  }

When I want to mock a result set that has only one row I use this:
   1:  private DbDataReader CreateDataReaderWithSingleResultSet(Dictionary<string, object> values,
   2:              int numberOfReads = 1, bool hasRows = true)
   3:  {
   4:      // get the basic fake database reader
   5:      var moqReader = CreateFakeDbDataReader(values == null ? 0 : values.Count, numberOfReads, hasRows: hasRows);
   6:   
   7:      if (values == null)
   8:      {
   9:          return moqReader.Object;
  10:      }
  11:   
  12:      // iterate the objects (fake data)
  13:      for (var i = 0; i < values.Count; i++)
  14:      {
  15:          var item = values.ElementAt(i);
  16:          // setup the reader to return the name of the "field" when the index is used on GetName
  17:          moqReader.Setup(p => p.GetName(i)).Returns(item.Key);
  18:          // setup the reader to return the value when it encounters the key
  19:          // this is where we specify that if reader["FirstName"] is evaluated, "Fake" (or whatever) will be returned
  20:          moqReader.SetupGet(p => p[item.Key]).Returns(item.Value);
  21:          moqReader.SetupGet(p => p[i]).Returns(item.Value);
  22:      }
  23:   
  24:      return moqReader.Object;
  25:  }

And when I want to mock a result set that has multiple rows I use this:

   1:  private DbDataReader CreateDataReaderWithSingleResultSetWithMultipleRows(List<Dictionary<string, object>> rows,
   2:              bool hasRows = true)
   3:  {
   4:      // get the basic fake database reader
   5:      var moqReader = CreateFakeDbDataReader(rows.First().Count, rows.Count, hasRows: hasRows);
   6:   
   7:      var currentRow = 0;
   8:   
   9:      // iterate through the "rows" of data
  10:      for (var i = 0; i < rows.Count; i++)
  11:      {
  12:          // for each "row" in the data, check if the current row is being retrieved
  13:          if (i != currentRow)
  14:          {
  15:              continue;
  16:          }
  17:   
  18:          var row = rows[i];
  19:          // iterate through the "fields" in the current row
  20:          for (var j = 0; j < row.Count; j++)
  21:          {
  22:              var item = row.ElementAt(j);
  23:              // setup the reader to return the name of the "field" when the index is used on GetName
  24:              moqReader.Setup(p => p.GetName(j)).Returns(item.Key);
  25:              // setup the reader to return the value when it encounters the key
  26:              // this is where we specify that if reader["FirstName"] is evaluated, "Fake" (or whatever) will be returned
  27:              moqReader.SetupGet(p => p[item.Key]).Returns(item.Value);
  28:          }
  29:      }
  30:   
  31:      return moqReader.Object;
  32:  }

The one part that I never got coded because I didn't really need to was returning multiple result sets.  I'm sure it can be done, but I haven't had to do it yet.  Happy coding!

Monday, January 18, 2016

Boot Camp

Let me start this post by saying that I'm not an Apple person.  I have a Windows phone (though I hate it; I prefer Android, but this one was cheaper) and Windows PCs at home and work.  I'm primarily a .NET developer (which you'll note is a Microsoft product).  I don't hate Macs (well, I didn't before this); I just don't use them.  They're more expensive than PCs and I don't know them as well as I know PCs.  I've had a PC since back when they were called IBM compatibles.

TL;DR: Boot Camp Assistant is probably great, but on old machines it sucks.  Snow Leopard with Boot Camp Assistant 3.2 is not as user friendly as newer versions probably are.  A Macbook Pro manufactured in 2009 cannot boot from a thumb drive.  When all else fails, just let the computer do it's thing.

All that said, I married an Apple person.  Everyone in her family is an Apple person.  I knew it when we decided to get married, but I didn't think it would be the one thing that finally made me crazy.  My mother-in-law got a new job, which requires her to have Windows on her home computer.  She has a Mac.  No problem, though, right?  Using Boot Camp Assistant, Mac will guide you through the painless process of dual booting OS X and Windows on the same machine.

I verified the specs, double-checked everything and got started.  It is now three days later and I'm almost finished.  Let me walk you through my troubles in the hopes that my trials and tribulations may help you in a similar situation.

I started off by launching Boot Camp Assistant and trying to download the drivers, as the instructions from Apple say to do.  No joy there.  A message came up that no drivers were found or something.  I honestly don't remember now; that was two days ago.  OK, no big deal.  A quick search of the Internet found the instructions that say I should just skip that part.  Got it.  Moving on then.  The next part of the wizard creates the partition for Windows.  At this point I'm happy.  I just want to make that clear.  At this point everything seems to be going according to plan.  The Chiefs are finally starting to put up a fight against the Patriots and it appears for the moment as though Apple's instructions will work fine.

But no.  Boot Camp Assistant can't create the partition because "some files could not be moved".  I find a video on YouTube where a guy explains how to get around it by running (what appear to be) some Unix commands.  That doesn't work either.  Some more searching of the Web and I find a solution to create a backup, wipe the drive, restore from backup and try again.  Well, that's not going to work for me.  I don't have an external hard drive to which to backup the OS and I don't particularly want to wipe the drive anyway.  I text the MIL and tell her no can do, boss.  She says I should go ahead and wipe it and reinstall.  Apparently she had anticipated this would happen and had backed everything up except her pictures.

OK.  Copy the pictures to a thumb drive, wipe the hard drive, copy the pictures back.  Got it.  How do I copy the pictures to the thumb drive?  It is now 11:30 PM on day 2.  I'm tired.  I'm falling asleep.  My wife is asleep next to me on the couch.  I want to quit.  OK.  Back to the Internet.  Ah, I need to "export" the files from iPhoto to the thumb drive.  Got it.  That was easy.  Time to wipe the hard drive.  Put the disc in and... hey, look at that... something went like it was supposed to.  Sweet.  I let it finish wiping and call it a night (although it is technically morning).

Surprisingly the reinstall goes pretty smoothly.  I will give Apple credit on this one: their install process is much faster than Windows (or so I thought at the time).  Great!  OK, OS X is installed.  Run Boot Camp Assistant again, partition the hard drive... and I'm done with the Mac side of things!  Woohoo!!

Getting to this point was supposed to be the hard part.  I had never done that stuff before.  From here on out it's just a matter of installing Windows, which I've done roughly a billion times.  Sweet.  I've got my bootable USB with a 32-bit copy of Windows 7 on it.  I've tested it to make sure it registers properly on my computer and we're good to go.  Put the thumb drive in when prompted for the disc and... Boot Camp Assistant doesn't see it.  Nor does it have any way to point to it.  Back to the Internet.  I see that I can install rEFIt and that should solve the problem.  Did that.  No joy.

After a lot more searching and a little bit of crying I decide I'll need to burn a DVD with Windows on it.  I don't have any blank DVDs.  To Walmart!  Back.  I make a bootable DVD.  I put it in the drive on the Mac.  I reboot the Mac and... nothing happens.  It just boots into OS X.  Oh, I forgot to hold the option key while it was booting.  No problem.  Reboot, hold option key (after the chime) and I get an option to boot to the Windows CD!  WOOOHOOO!!!  Choose that option and I'm home free... hahahahahahahahahahaha... no, that's not how it goes.

I end up on a black screen with a blinking cursor.  I wait.  I wait.  I wait.  I cry.  I wait some more.  I decide waiting isn't working.  Back to the Internet.  First suggestion: just wait.  Seriously.  Reboot into OS X.  Run Boot Camp Assistant.  Try to use the wizard.  It looks like it works, but I'm not going to get excited.  No.  I'm not going to.  Nope.  Nice try, Mac.  OK, what's this?  It looks like it's working!?  YES!!!!   Oh, wait, actually no.  It didn't work.

Reboot, hold option, choose Windows, black screen with blinking cursor, cry, yell, cry more, power down, walk away, drink, drink more, yell more, cry a little bit more, boot computer, forget to hold option... Windows is loading.  Holy crap.  No, this time I'm really not going to get excited.  I'm just going to stare at this screen through now-bloodshot eyes and wonder when this nightmare will end.  Wait, what's this?  It wants a product key?  I'm done?  HOLY CRAP IT WORKED!  I'M BRILLIANT!

Tuesday, December 1, 2015

Visual Studio Multiple Monitors

I've been using Visual Studio 2012 for some time now (yes, I'm aware that 2015 is already out; sometimes you don't have a choice) and it's been bugging me that when I split a "raft" out onto my second monitor it doesn't behave like a new instance of VS.  It only bugs me because it works that way for a bunch of other people and my old system worked that way.

Today I finally found the option to make it work the way I want it to.  Tools > Options > Environment > Tabs and Windows > Tab Well > Uncheck "Floating tab wells always stay on top of the main window".

This is so much better.

Tuesday, September 29, 2015

Searching all Fields in all Tables in all Databases in SQL Server

Sometimes you just want to push the limits of what you think you can do with a particular technology.  This was one of those times.  I wanted to see if there was an easy way to search every field in every table in every database on a server for a particular value.  Here's what I came up with.  Before anyone goes crazy (if anyone is actually reading this), I copied bits from built-in stored procedures SP_MSFOREACHDB and SP_MSFOREACH_WORKER to make this work.

I really don't like building the dynamic SQL like this, but it did get the job done so I guess I can't really complain about it.

DECLARE
     @DatabaseName NVARCHAR(517)
    ,@SearchText NVARCHAR(1000) = 'Some value'

DECLARE DatabaseCursor CURSOR GLOBAL FOR
SELECT [name]
FROM master.dbo.sysdatabases d
WHERE
    d.[status] &292 = 0
    AND DATABASEPROPERTY(d.[name], 'issingleuser') = 0
    AND HAS_DBACCESS(d.[name]) = 1
    AND [name] NOT IN ('master''model''msdb''tempdb')

OPEN DatabaseCursor
FETCH NEXT FROM DatabaseCursor INTO @DatabaseName

WHILE @@FETCH_STATUS = 0
BEGIN

    DECLARE @SQL NVARCHAR(MAX)

    SET @SQL = N'USE ' N'[' + @DatabaseName + N'] '
    SET @SQL = @SQL + N'DECLARE @SchemaName NVARCHAR(256), @TableName NVARCHAR(256), @ColumnName NVARCHAR(256) '
    SET @SQL = @SQL + N'DECLARE TableCursor CURSOR FOR '
    SET @SQL = @SQL + N'SELECT sch.[Name] AS SchemaName, st.[Name] AS TableName, sc.[Name] AS ColumnName FROM sys.tables st WITH (NOLOCK) INNER JOIN sys.columns sc WITH (NOLOCK) ON st.object_id = sc.object_id INNER JOIN sys.schemas sch WITH (NOLOCK) ON st.schema_id = sch.schema_id '
    SET @SQL = @SQL + N'OPEN TableCursor FETCH NEXT FROM TableCursor INTO @SchemaName, @TableName, @ColumnName '
    SET @SQL = @SQL + N'WHILE @@FETCH_STATUS = 0 BEGIN '
    SET @SQL = @SQL + N'DECLARE @InternalSQL NVARCHAR(MAX) = ''SELECT @CountParam = COUNT(*) FROM ['' + @SchemaName + ''].['' + @TableName + ''] WHERE ['' + @ColumnName + ''] LIKE ''''%' + @SearchText + '%'''''' '
    SET @SQL = @SQL + N'DECLARE @Count INT '
    SET @SQL = @SQL + N'EXEC SP_EXECUTESQL @InternalSQL, N''@CountParam INT OUT'', @Count OUT '
    SET @SQL = @SQL + N'IF (@Count > 0) '
    SET @SQL = @SQL + N'BEGIN '
    SET @SQL = @SQL + N'PRINT @SchemaName + ''.'' + @TableName + ''.'' + @ColumnName '
    SET @SQL = @SQL + N'END '
    SET @SQL = @SQL + N'FETCH NEXT FROM TableCursor INTO @SchemaName, @TableName, @ColumnName '
    SET @SQL = @SQL + N'END '
    SET @SQL = @SQL + N'CLOSE TableCursor '
    SET @SQL = @SQL + N'DEALLOCATE TableCursor '

    EXEC (@SQL)

    FETCH NEXT FROM DatabaseCursor INTO @DatabaseName

END

CLOSE DatabaseCursor
DEALLOCATE DatabaseCursor

Wednesday, September 23, 2015

ANSI_WARNINGS Problems

I've got a set of stored procedures that work together to accomplish a single goal.  I did things that way to make everything easier to unit test, and it worked like a charm.

Unfortunately, during actual testing a really strange error started occurring.  The last stored procedure called in the process (it's called by the "master" stored procedure) wasn't actually executing.  I added logs in everywhere and couldn't figure it out until a coworker suggested wrapping it in a TRY/CATCH.  Doing that exposed the issue, this guy: UPDATE failed because the following SET options have incorrect settings: 'ANSI_WARNINGS'. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or filtered indexes and/or query notifications and/or XML data type methods and/or spatial index operations.

Does that make sense to you?  No?  Good, I don't feel so bad now.  It didn't make sense to me either.  Fortunately, I work with an awesome team and one of the database guys figured it out pretty quickly.  It turns out that the stored procedure that wasn't getting called (we'll refer to it as "dbo.UpdateRecords") needed to have ANSI_WARNINGS set to ON: SET ANSI_WARNINGS ON.  Additionally, the calling stored procedure (we'll refer to it as "dbo.DoItAll") needed to set ANSI_WARNINGS to on right before calling dbo.UpdateRecords.  So the end result is like this:

SET ANSI_NULLS ON
GO
SET ANSI_WARNINGS OFF
GO
CREATE PROCEDURE
 dbo.DoItAll
AS
BEGIN
    ...
    SET ANSI_WARNINGS ON
    EXEC 
dbo.UpdateRecords
    ...
END
SET ANSI_NULLS, ANSI_WARNINGS ON
GO
CREATE PROCEDURE
 dbo.UpdateRecords
AS
BEGIN 

...
END

Friday, September 4, 2015

Unit Testing in SQL Server (Part 2a)

I mentioned in a previous post that I had written a bunch of tests to fully test the CustOrderHist stored procedure included in the Northwind database.  It turns out there were only two more.  Here they are:
CREATE PROCEDURE [CustOrderHist.Tests].[test Should Return Correct Quantity For Customer And Product]
AS
BEGIN

    -- Arrange
    -- Fake data insert statements moved to Setup stored procedure

    CREATE TABLE #Results (ProductName NVARCHAR(80), QuantityINT)

    -- Act
    INSERT INTO #Results
    EXEC dbo.CustOrderHist 'ABCDE'

    -- Assert
    DECLARE @Total INT
    SELECT @Total = QuantityFROM #Results WHERE ProductName = 'First Product'

    EXEC tSQLt.AssertEquals 513, @Total

END

CREATE PROCEDURE [CustOrderHist.Tests].[test Should Return Nothing For Customer With No Orders]
AS
BEGIN

    -- Arrange
    CREATE TABLE #Results (ProductName NVARCHAR(80), QuantityINT)

    -- Act
    INSERT INTO #Results
    EXEC dbo.CustOrderHist 'FGHIJ'

    -- Assert
    DECLARE @RowCount INT
    SELECT @RowCount = COUNT(*) FROM #Results

    EXECtSQLt.AssertEquals 0, @RowCount

END

Unit Testing in SQL Server (Part 2)

This is the second in a series of posts intended to get people up to speed using the tSQLt testing framework to test their stored procedures.  If you missed it, check out the first post here.  We'll be continuing where that one leaves off.

Now that we've got Northwind setup, we're ready to start learning what tSQLt does for us, and how it does it.  The main goal of tSQLt is to allow us to test our stored procedures.  Since stored procedures often rely on data and we can't really count on that data being in a specific state before our test, we're first going to learn how to setup our tests.

First up: test classes. A test class is just a schema in SQL Server. We use them to group our tests together. You should choose the best method for your organization when it comes to naming your test classes. At one client we decided to create a test class for each stored procedure. That way all of the tests for that stored procedure could be easily contained within a single test class. If something changed in the stored procedure we'd only have to modify the tests that were in that test class. I'll be following that convention here so I'm going to create a test class for all of the tests I write that test the functionality of dbo.CustOrderHist. I do that by running the following code on the Northwind database:

EXEC tSQLt.NewTestClass 'CustOrderHist.Tests'
Now that our test class exists we're ready to set up our data so we can test our stored procedure. When your tests run, tSQLt will first check the test class (in this case CustOrderHist.Tests) for a stored procedure named "setup" (case insensitive). If such a stored procedure is found, it is executed before each test in the test class. That makes it a great place to start setting up our data for our tests.

First, let's create the Setup procedure:

CREATE PROCEDURE [CustOrderHist.Tests].[Setup]
AS
BEGIN
END
GO
By examining CustOrderHist we can see that it relies on data from the Products, [Order Details], Orders, and Customers tables. Since we rely on that data, we'll need to set up that data before each of our tests run. Fortunately, tSQLt makes that really easy for us. We just have to execute tSQLt.FakeTable for each table. Here's the code:

CREATE PROCEDURE [CustOrderHist.Tests].[Setup]
AS
BEGIN

    EXEC tSQLt.FakeTable 'Products'
    EXEC tSQLt.FakeTable 'Order Details'
    EXEC tSQLt.FakeTable 'Orders'
    EXEC tSQLt.FakeTable 'Customers'

END


The FakeTable procedure opens a transaction, renames the table being passed, then recreates a table with the same name and structure, but without any constraints, defaults, or triggers. After the code above runs, all four tables will be empty shells of their normal selves, allowing us to populate whatever data we want into them.

The stored procedure we're testing is pretty basic so that's all the setup we're going to need.  Remember that from here on out all of the tests that we create in the CustOrderHist.Tests schema (test class) will first execute those FakeTable calls.  Now we're ready to create our first test.  For reference, this is what the CustOrderHist stored procedure actually looks like:


SELECT ProductName, Total=SUM(Quantity)
FROM Products P, [Order Details] OD, Orders O, Customers C
WHERE C.CustomerID = @CustomerID
AND C.CustomerID = O.CustomerID AND O.OrderID = OD.OrderID AND OD.ProductID = P.ProductID
GROUP BY ProductName

The first thing we're going to test is whether we get any results when we pass in a customer id that should have results.  Since a test in tSQLt is just a stored procedure we can create a test pretty darn easily.  Here's what it looks like before we actually have any contents in our test:


CREATE PROCEDURE [CustOrderHist.Tests].[test Should Return Results When Records Exist For Customer]
AS
BEGIN
END


As long as the stored procedure name starts with "test", tSQLt will pick it up as a test when the tests run.  Now that we have the test technically created, let's actually test something, shall we?  In our setup procedure we faked four tables.  Now we're going to populate those tables with some fake data to prove that our procedure under test does what we think it does:

CREATE PROCEDURE [CustOrderHist.Tests].[test Should Return Results When Records Exist For Customer]
AS
BEGIN

    -- Arrange
    INSERT INTO dbo.Products (ProductID, ProductName)
    VALUES (1, 'First Product'), (2, 'Second Product'), (3, 'Third Product')

    INSERT INTO dbo.Customers (CustomerID)
    VALUES ('ABCDE'), ('FGHIJ'), ('KLMNO'), ('PQRST')

    INSERT INTO dbo.Orders (OrderID, CustomerID)
    VALUES (1, 'ABCDE'), (2, 'ABCDE'), (3, 'PQRST'), (4, 'ABCDE')

    INSERT INTO dbo.[Order Details] (OrderID, ProductID, Quantity)
    VALUES (1, 1, 10), (2, 3, 100), (3, 2, 1000), (4, 1, 503)

    CREATE TABLE #Results (ProductName NVARCHAR(80), Quantity INT)

    -- Act

    -- Assert
END

You can see that in addition to populating fake data, I also created a temp table, which I'll use to store my results.  Now that we're all setup, we can go ahead and execute the stored procedure, putting the results into the temp table:

CREATE PROCEDURE [CustOrderHist.Tests].[test Should Return Results When Records Exist For Customer]
AS
BEGIN

    -- Arrange
    INSERT INTO dbo.Products (ProductID, ProductName)
    VALUES (1, 'First Product'), (2, 'Second Product'), (3, 'Third Product')

    INSERT INTO dbo.Customers (CustomerID)
    VALUES ('ABCDE'), ('FGHIJ'), ('KLMNO'), ('PQRST')

    INSERT INTO dbo.Orders (OrderID, CustomerID)
    VALUES (1, 'ABCDE'), (2, 'ABCDE'), (3, 'PQRST'), (4, 'ABCDE')

    INSERT INTO dbo.[Order Details] (OrderID, ProductID, Quantity)
    VALUES (1, 1, 10), (2, 3, 100), (3, 2, 1000), (4, 1, 503)

    CREATE TABLE #Results (ProductName NVARCHAR(80), Quantity INT)

    -- Act
    INSERT INTO #Results
    EXEC dbo.CustOrderHist 'ABCDE'

    -- Assert
END

The last step is to check whether what we have in our results temp table is what we expect to have there:

CREATE PROCEDURE [CustOrderHist.Tests].[test Should Return Results When Records Exist For Customer]
AS
BEGIN

    -- Arrange
    INSERT INTO dbo.Products (ProductID, ProductName)
    VALUES (1, 'First Product'), (2, 'Second Product'), (3, 'Third Product')

    INSERT INTO dbo.Customers (CustomerID)
    VALUES ('ABCDE'), ('FGHIJ'), ('KLMNO'), ('PQRST')

    INSERT INTO dbo.Orders (OrderID, CustomerID)
    VALUES (1, 'ABCDE'), (2, 'ABCDE'), (3, 'PQRST'), (4, 'ABCDE')

    INSERT INTO dbo.[Order Details] (OrderID, ProductID, Quantity)
    VALUES (1, 1, 10), (2, 3, 100), (3, 2, 1000), (4, 1, 503)

    CREATE TABLE #Results (ProductName NVARCHAR(80), Quantity INT)

    -- Act
    INSERT INTO #Results
    EXEC dbo.CustOrderHist 'ABCDE'

    -- Assert
    DECLARE @RowCount INT
    SELECT @RowCount = COUNT(*) FROM #Results

    EXEC tSQLt.AssertEquals 2, @RowCount

END

That last line runs a tSQLt procedure that compares the first parameter (expected result) to the second parameter (actual result). In this test we only wanted to see that the number of rows returned from the stored procedure was what we'd expect, and it was.
That's our first test. You run it like this:

EXEC tSQLt.Run CustOrderHist.Tests'

In order to fully test the CustOrderHist stored procedure, though, we'd need to write a few more tests. I've gone ahead and just written them and posted them here to give you an idea of what they would look like.