Finding Data in the Database

It's sometime useful to search an entire database to find out where some specific data is... for instance, you entered data into the UI but you aren't sure where it got stored.

This post contains a SQL Script to search a database for any specific string in any column in any table in your database.

To use this script, simply edit the first line to set the @value variable to the string you wish to search for (see the comment).  You can include "%" as the standard SQL Wildcard character.  As is, the script will search for any string starting with "C:\" ... but you can use exact strings or any arbitrary string you wish.

I hope you find this useful:


DECLARE @tableName sysname
DECLARE @columnName sysname
DECLARE @value varchar(100)
DECLARE @sql varchar(2000)
DECLARE @sqlPreamble varchar(100)

SET @value = 'C:%' -- *** <-- Set this to the value you're searching for *** --

SET @sqlPreamble = 'IF EXISTS (SELECT 1 FROM '

DECLARE theTableCursor CURSOR FAST_FORWARD FOR 
    SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES
     WHERE TABLE_SCHEMA = 'dbo' AND TABLE_TYPE = 'BASE TABLE' 
       AND TABLE_NAME NOT LIKE '%temp%' AND TABLE_NAME != 'dtproperties' AND TABLE_NAME != 'sysdiagrams'
     ORDER BY TABLE_NAME

OPEN theTableCursor
FETCH NEXT FROM theTableCursor INTO @tableName

WHILE @@FETCH_STATUS = 0 -- spin through Table entries
BEGIN
    DECLARE theColumnCursor CURSOR FAST_FORWARD FOR
        SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS
         WHERE TABLE_NAME = @tableName AND (DATA_TYPE = 'nvarchar' OR DATA_TYPE = 'varchar')
         ORDER BY ORDINAL_POSITION

    OPEN theColumnCursor
    FETCH NEXT FROM theColumnCursor INTO @columnName

    WHILE @@FETCH_STATUS = 0 -- spin through Column entries
    BEGIN
        SET @sql = @tableName + ' WHERE ' + @columnName + ' LIKE ''' + @value + 
                   ''') PRINT ''Value found in Table: ' + @tableName + ', Column: ' +  @columnName + ''''
        EXEC (@sqlPreamble + @sql)
        FETCH NEXT FROM theColumnCursor INTO @columnName
    END
    CLOSE theColumnCursor
    DEALLOCATE theColumnCursor

    FETCH NEXT FROM theTableCursor INTO @tableName
END
CLOSE theTableCursor
DEALLOCATE theTableCursor

 


 The output of this script will be the table name, and column name, where the data was found.  You can then do a SELECT on that table to see which row(s) contain the data.  A sample output looks like this:

Value found in Table: Publish_Server_Ref, Column: SourceRootLocation
Value found in Table: PublishServerRef, Column: LocalPublishPath
Value found in Table: System_Params, Column: ShortValue
Value found in Table: System_Variable, Column: Value
Value found in Table: System_Variable, Column: DefaultValue
Value found in Table: SystemConfig, Column: ParameterValue
Value found in Table: Website, Column: PublishRootPath
Value found in Table: Website, Column: ProtectedRootPath

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

I do!

Thanks for posting this.  These quick examples are great.  I'm incorporating this into our standard "sql admin" script now.

If I might make a suggestion, since this is a firehose "look everywhere" type of procedure, it might be wise to add a strategic (nolock) to the generated SQL.  Normally, I avoid nolock (locking is good and beneficial) but this scenario looks like it might justify it, especially in a large database.

SET @sql = @tableName + ' (nolock) WHERE ' + @columnName + ' LIKE ''' + @value + 

--
Bruce Wilson
Director, Technical Services
RSM McGladrey, Inc.

Another refinement

In a large database, it might be very costly to spend time checking the fields which are shorter than the provided pattern, so I added this code to figure out the minimum field width, right under "set @value = 'C:%' ":

    declare @minlength int,
        @tempvalue varchar(100),
        @openbracket int,
        @closebracket int
   
    set @minlength = len(@value)
    set @tempvalue = @value
   
    -- Bracketed lists always match one character, so replace with one character
    set @openbracket = charindex('[', @tempvalue)
    set @closebracket = charindex(']', @tempvalue)

    while @openbracket > 0 and @closebracket > @openbracket
    begin
        set @tempvalue = stuff(@tempvalue, @openbracket, (@closebracket - @openbracket + 1), '_')
        set @openbracket = charindex('[', @tempvalue)
        set @closebracket = charindex(']', @tempvalue)
    end

    -- % wildcards will match zero length, so they can all be removed
    set @tempvalue = replace(@tempvalue, '%', '')

    set @minlength = len(@tempvalue)

    if @minlength = 0
    begin
        print 'Cannot search for empty string or only % wildcards.'
        return
    end

 

 

And then changed the columnCursor like this:

        DECLARE theColumnCursor CURSOR FAST_FORWARD FOR
            SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS
            WHERE TABLE_NAME = @tableName
                AND DATA_TYPE in ('nvarchar', 'varchar')
                and CHARACTER_MAXIMUM_LENGTH >= @minlength
            ORDER BY ORDINAL_POSITION

 

--
Bruce Wilson
Director, Technical Services
RSM McGladrey, Inc.

Interesting!  The string

Interesting!  The string length optimization is an obvious one (I thought about it but just never got around to adding it, because I'm rarely working on a live server), and a good one.  I honestly can say I never searched for strings with embedded square brackets, so escaping those (or other potential escape characters) never actually occured to me.  Good stuff!

For the record, I have three very similar scripts... one finds integers, one finds GUIDs, and the final one finds columns that have datetimes that fall into a specified range.  I haven't yet needed one that finds decimal values within a specific range, but when I do, that's easy enough to copy and tweak as well.  They're all very close variations on a theme (which is another reason I don't have type-specific optimizations in this one... the string one actually grew out of the GUID one, which is what I needed at first, and thus wrote first).

 

Brackets are helpful

I like brackets as a poor-man's regex.  For a (hopefully) familiar example, you can identify non-routable IP addresses using these patterns:

10.%[0-9].%[0-9].%[0-9]     (10.x.x.x)
169.254.%[0-9].%[0-9]       (169.254.x.x)
172.1[6-9].%[0-9].%[0-9]    (172.16.x.x thru 172.19.x.x)
172.2[0-3].%[0-9].%[0-9]    (172.20.x.x thru 172.23.x.x)
192.168.%[0-9].%[0-9]       (192.168.x.x)
127.0.0.%[0-9]                     (127.0.0.x)

It's not perfect, but it's an improvement over a simple % wildcard, and it uses a string matching algorithm instead of a large table (10.0.0.0, 10,0.0.1... 10.255.255.255), or a long "in" statement, or a long list of "or" tests.  Not appropriate for every situation, but one more tool in the arsenal.

I suppose if you ignored the bracket handling, you'd just search more fields, but still less than checking them all, so it wouldn't be the end of the world.  And of course, my logic for collapsing the brackets is also imperfect -- it won't catch mismatched or out-of-order brackets.

--
Bruce Wilson
Director, Technical Services
RSM McGladrey, Inc.

Last one

TABLE_NAME not like '%temp%' will exclude any table with Template in its name.  iMIS 15.2 has 6 such tables.
--
Bruce Wilson
Director, Technical Services
RSM McGladrey, Inc.