Tuesday, September 29, 2015

Searching all Fields in all Tables in all Databases in SQL Server

Sometimes you just want to push the limits of what you think you can do with a particular technology.  This was one of those times.  I wanted to see if there was an easy way to search every field in every table in every database on a server for a particular value.  Here's what I came up with.  Before anyone goes crazy (if anyone is actually reading this), I copied bits from built-in stored procedures SP_MSFOREACHDB and SP_MSFOREACH_WORKER to make this work.

I really don't like building the dynamic SQL like this, but it did get the job done so I guess I can't really complain about it.

DECLARE
     @DatabaseName NVARCHAR(517)
    ,@SearchText NVARCHAR(1000) = 'Some value'

DECLARE DatabaseCursor CURSOR GLOBAL FOR
SELECT [name]
FROM master.dbo.sysdatabases d
WHERE
    d.[status] &292 = 0
    AND DATABASEPROPERTY(d.[name], 'issingleuser') = 0
    AND HAS_DBACCESS(d.[name]) = 1
    AND [name] NOT IN ('master''model''msdb''tempdb')

OPEN DatabaseCursor
FETCH NEXT FROM DatabaseCursor INTO @DatabaseName

WHILE @@FETCH_STATUS = 0
BEGIN

    DECLARE @SQL NVARCHAR(MAX)

    SET @SQL = N'USE ' N'[' + @DatabaseName + N'] '
    SET @SQL = @SQL + N'DECLARE @SchemaName NVARCHAR(256), @TableName NVARCHAR(256), @ColumnName NVARCHAR(256) '
    SET @SQL = @SQL + N'DECLARE TableCursor CURSOR FOR '
    SET @SQL = @SQL + N'SELECT sch.[Name] AS SchemaName, st.[Name] AS TableName, sc.[Name] AS ColumnName FROM sys.tables st WITH (NOLOCK) INNER JOIN sys.columns sc WITH (NOLOCK) ON st.object_id = sc.object_id INNER JOIN sys.schemas sch WITH (NOLOCK) ON st.schema_id = sch.schema_id '
    SET @SQL = @SQL + N'OPEN TableCursor FETCH NEXT FROM TableCursor INTO @SchemaName, @TableName, @ColumnName '
    SET @SQL = @SQL + N'WHILE @@FETCH_STATUS = 0 BEGIN '
    SET @SQL = @SQL + N'DECLARE @InternalSQL NVARCHAR(MAX) = ''SELECT @CountParam = COUNT(*) FROM ['' + @SchemaName + ''].['' + @TableName + ''] WHERE ['' + @ColumnName + ''] LIKE ''''%' + @SearchText + '%'''''' '
    SET @SQL = @SQL + N'DECLARE @Count INT '
    SET @SQL = @SQL + N'EXEC SP_EXECUTESQL @InternalSQL, N''@CountParam INT OUT'', @Count OUT '
    SET @SQL = @SQL + N'IF (@Count > 0) '
    SET @SQL = @SQL + N'BEGIN '
    SET @SQL = @SQL + N'PRINT @SchemaName + ''.'' + @TableName + ''.'' + @ColumnName '
    SET @SQL = @SQL + N'END '
    SET @SQL = @SQL + N'FETCH NEXT FROM TableCursor INTO @SchemaName, @TableName, @ColumnName '
    SET @SQL = @SQL + N'END '
    SET @SQL = @SQL + N'CLOSE TableCursor '
    SET @SQL = @SQL + N'DEALLOCATE TableCursor '

    EXEC (@SQL)

    FETCH NEXT FROM DatabaseCursor INTO @DatabaseName

END

CLOSE DatabaseCursor
DEALLOCATE DatabaseCursor

Wednesday, September 23, 2015

ANSI_WARNINGS Problems

I've got a set of stored procedures that work together to accomplish a single goal.  I did things that way to make everything easier to unit test, and it worked like a charm.

Unfortunately, during actual testing a really strange error started occurring.  The last stored procedure called in the process (it's called by the "master" stored procedure) wasn't actually executing.  I added logs in everywhere and couldn't figure it out until a coworker suggested wrapping it in a TRY/CATCH.  Doing that exposed the issue, this guy: UPDATE failed because the following SET options have incorrect settings: 'ANSI_WARNINGS'. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or filtered indexes and/or query notifications and/or XML data type methods and/or spatial index operations.

Does that make sense to you?  No?  Good, I don't feel so bad now.  It didn't make sense to me either.  Fortunately, I work with an awesome team and one of the database guys figured it out pretty quickly.  It turns out that the stored procedure that wasn't getting called (we'll refer to it as "dbo.UpdateRecords") needed to have ANSI_WARNINGS set to ON: SET ANSI_WARNINGS ON.  Additionally, the calling stored procedure (we'll refer to it as "dbo.DoItAll") needed to set ANSI_WARNINGS to on right before calling dbo.UpdateRecords.  So the end result is like this:

SET ANSI_NULLS ON
GO
SET ANSI_WARNINGS OFF
GO
CREATE PROCEDURE
 dbo.DoItAll
AS
BEGIN
    ...
    SET ANSI_WARNINGS ON
    EXEC 
dbo.UpdateRecords
    ...
END
SET ANSI_NULLS, ANSI_WARNINGS ON
GO
CREATE PROCEDURE
 dbo.UpdateRecords
AS
BEGIN 

...
END

Friday, September 4, 2015

Unit Testing in SQL Server (Part 2a)

I mentioned in a previous post that I had written a bunch of tests to fully test the CustOrderHist stored procedure included in the Northwind database.  It turns out there were only two more.  Here they are:
CREATE PROCEDURE [CustOrderHist.Tests].[test Should Return Correct Quantity For Customer And Product]
AS
BEGIN

    -- Arrange
    -- Fake data insert statements moved to Setup stored procedure

    CREATE TABLE #Results (ProductName NVARCHAR(80), QuantityINT)

    -- Act
    INSERT INTO #Results
    EXEC dbo.CustOrderHist 'ABCDE'

    -- Assert
    DECLARE @Total INT
    SELECT @Total = QuantityFROM #Results WHERE ProductName = 'First Product'

    EXEC tSQLt.AssertEquals 513, @Total

END

CREATE PROCEDURE [CustOrderHist.Tests].[test Should Return Nothing For Customer With No Orders]
AS
BEGIN

    -- Arrange
    CREATE TABLE #Results (ProductName NVARCHAR(80), QuantityINT)

    -- Act
    INSERT INTO #Results
    EXEC dbo.CustOrderHist 'FGHIJ'

    -- Assert
    DECLARE @RowCount INT
    SELECT @RowCount = COUNT(*) FROM #Results

    EXECtSQLt.AssertEquals 0, @RowCount

END

Unit Testing in SQL Server (Part 2)

This is the second in a series of posts intended to get people up to speed using the tSQLt testing framework to test their stored procedures.  If you missed it, check out the first post here.  We'll be continuing where that one leaves off.

Now that we've got Northwind setup, we're ready to start learning what tSQLt does for us, and how it does it.  The main goal of tSQLt is to allow us to test our stored procedures.  Since stored procedures often rely on data and we can't really count on that data being in a specific state before our test, we're first going to learn how to setup our tests.

First up: test classes. A test class is just a schema in SQL Server. We use them to group our tests together. You should choose the best method for your organization when it comes to naming your test classes. At one client we decided to create a test class for each stored procedure. That way all of the tests for that stored procedure could be easily contained within a single test class. If something changed in the stored procedure we'd only have to modify the tests that were in that test class. I'll be following that convention here so I'm going to create a test class for all of the tests I write that test the functionality of dbo.CustOrderHist. I do that by running the following code on the Northwind database:

EXEC tSQLt.NewTestClass 'CustOrderHist.Tests'
Now that our test class exists we're ready to set up our data so we can test our stored procedure. When your tests run, tSQLt will first check the test class (in this case CustOrderHist.Tests) for a stored procedure named "setup" (case insensitive). If such a stored procedure is found, it is executed before each test in the test class. That makes it a great place to start setting up our data for our tests.

First, let's create the Setup procedure:

CREATE PROCEDURE [CustOrderHist.Tests].[Setup]
AS
BEGIN
END
GO
By examining CustOrderHist we can see that it relies on data from the Products, [Order Details], Orders, and Customers tables. Since we rely on that data, we'll need to set up that data before each of our tests run. Fortunately, tSQLt makes that really easy for us. We just have to execute tSQLt.FakeTable for each table. Here's the code:

CREATE PROCEDURE [CustOrderHist.Tests].[Setup]
AS
BEGIN

    EXEC tSQLt.FakeTable 'Products'
    EXEC tSQLt.FakeTable 'Order Details'
    EXEC tSQLt.FakeTable 'Orders'
    EXEC tSQLt.FakeTable 'Customers'

END


The FakeTable procedure opens a transaction, renames the table being passed, then recreates a table with the same name and structure, but without any constraints, defaults, or triggers. After the code above runs, all four tables will be empty shells of their normal selves, allowing us to populate whatever data we want into them.

The stored procedure we're testing is pretty basic so that's all the setup we're going to need.  Remember that from here on out all of the tests that we create in the CustOrderHist.Tests schema (test class) will first execute those FakeTable calls.  Now we're ready to create our first test.  For reference, this is what the CustOrderHist stored procedure actually looks like:


SELECT ProductName, Total=SUM(Quantity)
FROM Products P, [Order Details] OD, Orders O, Customers C
WHERE C.CustomerID = @CustomerID
AND C.CustomerID = O.CustomerID AND O.OrderID = OD.OrderID AND OD.ProductID = P.ProductID
GROUP BY ProductName

The first thing we're going to test is whether we get any results when we pass in a customer id that should have results.  Since a test in tSQLt is just a stored procedure we can create a test pretty darn easily.  Here's what it looks like before we actually have any contents in our test:


CREATE PROCEDURE [CustOrderHist.Tests].[test Should Return Results When Records Exist For Customer]
AS
BEGIN
END


As long as the stored procedure name starts with "test", tSQLt will pick it up as a test when the tests run.  Now that we have the test technically created, let's actually test something, shall we?  In our setup procedure we faked four tables.  Now we're going to populate those tables with some fake data to prove that our procedure under test does what we think it does:

CREATE PROCEDURE [CustOrderHist.Tests].[test Should Return Results When Records Exist For Customer]
AS
BEGIN

    -- Arrange
    INSERT INTO dbo.Products (ProductID, ProductName)
    VALUES (1, 'First Product'), (2, 'Second Product'), (3, 'Third Product')

    INSERT INTO dbo.Customers (CustomerID)
    VALUES ('ABCDE'), ('FGHIJ'), ('KLMNO'), ('PQRST')

    INSERT INTO dbo.Orders (OrderID, CustomerID)
    VALUES (1, 'ABCDE'), (2, 'ABCDE'), (3, 'PQRST'), (4, 'ABCDE')

    INSERT INTO dbo.[Order Details] (OrderID, ProductID, Quantity)
    VALUES (1, 1, 10), (2, 3, 100), (3, 2, 1000), (4, 1, 503)

    CREATE TABLE #Results (ProductName NVARCHAR(80), Quantity INT)

    -- Act

    -- Assert
END

You can see that in addition to populating fake data, I also created a temp table, which I'll use to store my results.  Now that we're all setup, we can go ahead and execute the stored procedure, putting the results into the temp table:

CREATE PROCEDURE [CustOrderHist.Tests].[test Should Return Results When Records Exist For Customer]
AS
BEGIN

    -- Arrange
    INSERT INTO dbo.Products (ProductID, ProductName)
    VALUES (1, 'First Product'), (2, 'Second Product'), (3, 'Third Product')

    INSERT INTO dbo.Customers (CustomerID)
    VALUES ('ABCDE'), ('FGHIJ'), ('KLMNO'), ('PQRST')

    INSERT INTO dbo.Orders (OrderID, CustomerID)
    VALUES (1, 'ABCDE'), (2, 'ABCDE'), (3, 'PQRST'), (4, 'ABCDE')

    INSERT INTO dbo.[Order Details] (OrderID, ProductID, Quantity)
    VALUES (1, 1, 10), (2, 3, 100), (3, 2, 1000), (4, 1, 503)

    CREATE TABLE #Results (ProductName NVARCHAR(80), Quantity INT)

    -- Act
    INSERT INTO #Results
    EXEC dbo.CustOrderHist 'ABCDE'

    -- Assert
END

The last step is to check whether what we have in our results temp table is what we expect to have there:

CREATE PROCEDURE [CustOrderHist.Tests].[test Should Return Results When Records Exist For Customer]
AS
BEGIN

    -- Arrange
    INSERT INTO dbo.Products (ProductID, ProductName)
    VALUES (1, 'First Product'), (2, 'Second Product'), (3, 'Third Product')

    INSERT INTO dbo.Customers (CustomerID)
    VALUES ('ABCDE'), ('FGHIJ'), ('KLMNO'), ('PQRST')

    INSERT INTO dbo.Orders (OrderID, CustomerID)
    VALUES (1, 'ABCDE'), (2, 'ABCDE'), (3, 'PQRST'), (4, 'ABCDE')

    INSERT INTO dbo.[Order Details] (OrderID, ProductID, Quantity)
    VALUES (1, 1, 10), (2, 3, 100), (3, 2, 1000), (4, 1, 503)

    CREATE TABLE #Results (ProductName NVARCHAR(80), Quantity INT)

    -- Act
    INSERT INTO #Results
    EXEC dbo.CustOrderHist 'ABCDE'

    -- Assert
    DECLARE @RowCount INT
    SELECT @RowCount = COUNT(*) FROM #Results

    EXEC tSQLt.AssertEquals 2, @RowCount

END

That last line runs a tSQLt procedure that compares the first parameter (expected result) to the second parameter (actual result). In this test we only wanted to see that the number of rows returned from the stored procedure was what we'd expect, and it was.
That's our first test. You run it like this:

EXEC tSQLt.Run CustOrderHist.Tests'

In order to fully test the CustOrderHist stored procedure, though, we'd need to write a few more tests. I've gone ahead and just written them and posted them here to give you an idea of what they would look like.