SQL to help validate 10.6 DB is ready to upgrade to 15.x

The following SQL script can be run on any 10.6 database to validate that some specific, common data problems that would interfere with a smooth upgrade to 15.x are not present. If any issues are present, they are displayed. If you run this from SQL Query Analyzer or SQL Server Management Studio, set the "results to text" option before running this script, for the best display of the results.

-------------------------------------------------------------------
-- Preparing to upgrade iMIS 10.6 to iMIS 15.x, Part I
-- 
-- Check for non-unique user logins
--
-- This script will check for any duplicate user ids or web logins
-- that would cause the database upgrade process to fail.  This is
-- a simple 'sanity' check.  Run this script and if any user ids or
-- web logins are identified as having duplicate entries, you will
-- need to correct those duplicates (provide unique logins for each
-- user) prior to upgrading to 15.x
--
-- Hint:  If using Query Analyser or SQL Server Management Studio,
--        use the "Results to Text" option to view the results.
-------------------------------------------------------------------
IF EXISTS ( SELECT [UserId], COUNT([UserId]) AS [Count]
              FROM [dbo].[Users]
             WHERE [UserId] <> ''
             GROUP BY [UserId]
            HAVING COUNT([UserId]) > 1 ) 
BEGIN
    PRINT 'Error: duplicate User IDs exist in the Users table:'
    SELECT [UserId], COUNT([UserId]) AS [Count]
      FROM [dbo].[Users]
     WHERE [UserId] <> ''
     GROUP BY [UserId]
    HAVING COUNT([UserId]) > 1
END
ELSE 
    PRINT 'The User IDs in the Users table are all unique.'

IF EXISTS ( SELECT [WEB_LOGIN], COUNT([WEB_LOGIN]) AS [Count]
              FROM [dbo].[Name_Security]
             WHERE [WEB_LOGIN] <> ''
             GROUP BY [WEB_LOGIN]
            HAVING COUNT([WEB_LOGIN]) > 1 )
BEGIN
    PRINT 'Error: duplicate Web Logins exist in the Name_Security table:'
    SELECT [WEB_LOGIN], COUNT([WEB_LOGIN]) AS [Count]
      FROM [dbo].[Name_Security]
     WHERE [WEB_LOGIN] <> ''
     GROUP BY [WEB_LOGIN]
    HAVING COUNT([WEB_LOGIN]) > 1
END
ELSE 
    PRINT 'The Web Logins in the Name_Security table are all unique.'

IF EXISTS ( SELECT 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 'UserMain' )
BEGIN
    IF EXISTS ( SELECT [UserId], COUNT([UserId]) AS [Count]
                  FROM [dbo].[UserMain]
                 WHERE [UserId] <> ''
                 GROUP BY [UserId]
                HAVING COUNT([UserId]) > 1 )
    BEGIN
        PRINT 'Error: duplicate User IDs exist in the UserMain table:'
        SELECT [UserId], COUNT([UserId]) AS [Count]
          FROM [dbo].[UserMain]
         WHERE [UserId] <> ''
         GROUP BY [UserId]
        HAVING COUNT([UserId]) > 1
    END
    ELSE 
        PRINT 'The User IDs in the UserMain table are all unique.'
END

If everything is fine, the output should look like this:

The User IDs in the Users table are all unique.
The Web Logins in the Name_Security table are all unique.
The User IDs in the UserMain table are all unique.

If there are any problems, the specific issues will be displayed.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Good stuff, need more like this

Thanks for putting this out here, Paul. I know there have to be more issues like this. Hopefully this can become a "sticky" topic where everyone can contribute their own discoveries.

I know of one that can affect the upgrade: the existence of a table named "Demo". If you have a table by this name, the upgrade will crash, as it first thinks the table already exists (so it won't try to create it), then references the fields within it (which likely don't match what iMIS expects). Tech Support has a semi-supported script which will remove references to Demo in all the releases and patches from 15.0 until the version where it was no longer included.

We discovered this for one client who used "Demo" as their primary custom demographics table. The number of references to the table under its old name (views, procedures, reports) made it prohibitive to simply rename the table.

The "Demo" table was mistakenly included in the iMIS release, but isn't needed by iMIS. I suspect it was created for some kind of testing, and nobody noticed it to remove it before the final build. There's one other table in this situation, but I don't recall the name of it, and I don't know whether it's included in the patch.

Suggested addition to the script:

IF EXISTS ( SELECT 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 'Demo' )
    PRINT 'Error: Database includes a table named Demo. Obtain patch from Tech Support before attempting upgrade.'
ELSE 
    PRINT 'No "Demo" table found, OK'
END

-- Bruce

Side note: This issue

Side note: This issue should be fixed in a 15.0.3 patch, and is definitely fixed in the upcoming version (15.1.x).

Side note: yes but

The "Demo" table is removed from a 15.0.3 version, but because you have to pass through the patches for previous versions on your way up to your final version, it can still cause a problem during an upgrade.

That is, unless there's been some effort since then to incorporate TechSupport's fix into the previous patches as they sit on the hard drive?

-- Bruce

Right... with the patch

Right... with the patch there is still an issue that has to be worked around (the initial install fails, apply the patch, restore the DB, then it should work). That work around isn't necessary going forward effective with the next release.

Opportunity Management Errors and more

I had upgrade errors while working with one customer who was using opportunity management and duplicate merge manager. They merged some records, and this merge did not update any of the opportunities. So the opportunity was referencing a user/prospect that didnt exist. I just deleted these opportunities, but you could modify the scripts to somehow update the user.

/*Make sure that every record in name has a name_security record*/
insert into name_security (ID)
select ID from name where id not in (select id from name_security)

/*
The delete statements below should resolve the following upgrade error:

ALTER TABLE statement conflicted with COLUMN FOREIGN KEY constraint
'FK_OpportunityMain_ContactMain'. The conflict occurred in database 'imis',
table 'ContactMain', column 'ContactKey'.
Msg 547, Level 16, State 1, Server AIMGEN1, Line 1

*/
delete from opportunitymainsales where opportunitykey in 
(select opportunitykey from opportunitymain where prospectkey in 
	(select userkey from usermain where contactmaster not in (select id from name))
)

delete from opportunitymainmember where opportunitykey in 
(select opportunitykey from opportunitymain where prospectkey in 
	(select userkey from usermain where contactmaster not in (select id from name))
)

delete from opportunitymain where prospectkey in 
	(select userkey from usermain where contactmaster not in (select id from name))

/*
The delete statements below should resolve the following upgrade error:

INSERT statement conflicted with COLUMN FOREIGN KEY constraint
'FK_GroupMember_GroupMain'. The conflict occurred in database 'iMIS', table
'GroupMain', column 'GroupKey'.
Msg 547, Level 16, State 1, Server DEVAIMIMIS, Line 33

*/

delete from opportunitymainsales where opportunitykey in 
(select opportunitykey from OpportunityMain where ContactGroupKey not in (select groupkey from groupmain)
)

delete from opportunitymainmember where opportunitykey in 
(select opportunitykey from OpportunityMain where ContactGroupKey not in (select groupkey from groupmain)
)

delete from OpportunityMain where ContactGroupKey not in (select groupkey from groupmain)

/*Run the update below to resolve the following upgrade error:

ALTER TABLE statement conflicted with COLUMN FOREIGN KEY constraint
'FK_TaskItem_UserMain_CompletedBy'. The conflict occurred in database 'imis',
table 'UserMain', column 'UserKey'.
Msg 547, Level 16, State 1, Server AIMGEN1, Line 1


Note: This script had to be run between step 45 and 57 of script 1 of the upgrade. 
For some reason during the upgrade process another userkey gets added to taskitem which is not in usermain.
So have to set this to null so that the upgrade does not fail.
*/
update taskitem
set completedbyuserkey = null
where completedbyuserkey not in (select userkey from usermain)
and completedbyuserkey is not null


/*
The following error occurs when transferring a database from one server to another server and then 
trying to run the iMIS 15 upgrade scripts. This happens when the CreateSqlLogins.sql script tries to drop the manager login.

*** ERROR creating SQL logins ***

Login 'MANAGER_iMIS' is aliased or mapped to a user in one or more
database(s). Drop the user or alias before dropping the login.
Msg 15175, Level 16, State 1, Server DEVAIMIMIS, Procedure sp_droplogin, Line 93

*** PROCESS ENDED WITH ERRORS !!! ***

*/

/* Run this select statement against each database to see if the MANAGER login exists.
If it exists and su.name = '\MANAGER_iMIS' then run the sp_dropalias command to remove it.
If it exists and su.name = 'MANAGER_iMIS' then run the sp_dropuser command to remove it.
Once it has been removed from all databases on the server, then run the sp_droplogin command.
*/
select su.name,sl.name,* from sysusers su inner join master..syslogins sl on su.sid = sl.sid

-- Run this against all databases i.e. dev and test
sp_dropalias 'MANAGER_iMIS'
sp_dropuser 'MANAGER_iMIS'
-- Then run this to drop the login
sp_droplogin 'MANAGER_iMIS'


I've been meaning to thank

I've been meaning to thank you for this input as well... very useful.

Obviously iMIS upgrade can't incorporate these 'fixes', as we don't ever want to just delete data like that. The best thing to do in such cases is to actually update the records to have them refer back to the correct user (the one merged), but that's not possible to determine programatically (and may even be difficult for a human to do unless they have copies of a pre-merge backup database to reference).

For the record, iMIS15.1 upgrade (from 10.6) will automatically fix up Name_Security and Name_Fin records, as you have in your very first statement.

However, it would be a very good idea for people to run variations on your statements dealing with opportunities... "SELECT" versions to ensure they don't have any orphaned rows that will cause upgrades to fail. Perhaps if I get some extra time, I'll work up some more sanity check scripts along those lines that can be run.

Thanks again!