Performance counter created but not working until windows restart

Question

I have a strange problem creating new counters in existing group. I have a windows service, which does some work and allow me to monitor its state via performance counter. I have performance counters group and some performance counters. Group is created during service install phase (with administrator privileges), and counters are initialized when service starts (as LocalSystem user). Everything works fine, group exists, counter exist too, I can monitor them and write to Performance counters log. Service is constantly running.

Then I want to add some more performance counters. I add them in service code, rebuild and deploy to server by uninstalling previous service instance (performance counters group is deleted by uninstall code), and installing service again (performance counters group is created during install phase with all new counters). Then I start service.

That's the end of common, uninteresting part of the story. The strange things start here.

I go to PerMon, add all counters to System monitor view. I can see my performance counters group, I see all performance counters including NEW performance counters that I've just added. I can add them to System monitor view. And I can see, that old counters are working. But new counters doesn't work, they don't collect any data. Well, ok, perhaps there was my mistake, I switch to log view and try to log performance counters data. Old counters are logged as they logged before. But when I try to add new counter, I find in Event Viewer following warning:

The service was unable to add the counter '\AGENT\MyCountersGroupName\MyNewCounter' to the NewCountersLog log or alert. This log or alert will continue, but data for that counter will not be collected. The error returned is: The specified counter could not be found.

I tried to reinstall service, remove old counters, add them again, and nothing changed. Old counters are working, and new counters don't work. Then I restart Windows and new counters started working! Nothing changed in service, I've just restarted the server. I encountered this problem on 2 servers, the both are running under Windows Server 2003 SP1. The code for all performance counters is identical, 'cause I create them using code with generics.

You can say "hey, don't bother, restart your Windows every time when you need to add new performance counters", but I can't. My service is running on server along with other services, and we need these services work constantly, we can't restart server every time I change one service.

Can anybody help with this issue?

Update: Looks like there's the similar problem: https://stackoverflow.com/questions/2180005/adding-performance-counters-to-existing-category

Update I don't think that the problem is in my code, but if it helps I post it here.

Code to install and uninstall (called during install phase): PerformanceCountersManagerBase.GetCreationData() is a generic wrapper for obtaining collection of data to create performance counters.

public static void Install()
{
    CounterCreationDataCollection counters = new CounterCreationDataCollection(
    PerformanceCountersManagerBase.GetCreationData<TManager>().ToArray());

    if (PerformanceCounterCategory.Exists(PerformanceCountersManagerBase.GroupName))
        PerformanceCounterCategory.Delete(PerformanceCountersManagerBase.

    PerformanceCounterCategory.Create(PerformanceCountersManagerBase.GroupName, 
        "Group description",
        PerformanceCounterCategoryType.SingleInstance, counters);
}

public static void Uninstall()
{
    if (!PerformanceCounterCategory.Exists(PerformanceCountersManagerBase.GroupName))
        return;

    PerformanceCounterCategory.Delete(PerformanceCountersManagerBase.GroupName);
}

Code to initialize counters in service and update:

internal void Initialize(string name, string groupName)
{
    _counter = new PerformanceCounter(groupName, name, false);
    _counter.RawValue = 0;
}

internal void Increment()
{
    if ((_counter == null) || _counter.ReadOnly)
        return;

    lock (_counter)
    {
        _counter.Increment();
    }
}

Update 3 I changed code to install counters via .NET component PerformanceCounterInstaller, and nothing changed. Old counters work as they worked, and newly created don't work and attempt to log them results in the exact error (warning) message in event log. The installer creation code is as follows:

public static PerformanceCounterInstaller GetInstaller()
{
    PerformanceCounterGroupAttribute group
        = PerformanceCountersManagerBase.ExtractGroupSettings(typeof(TManager));

    PerformanceCounterInstaller installer = new PerformanceCounterInstaller();
    installer.CategoryName = group.Name;
    installer.CategoryHelp = group.Description;
    installer.CategoryType = PerformanceCounterCategoryType.SingleInstance;
    installer.UninstallAction = UninstallAction.Remove;
    installer.Counters.AddRange(PerformanceCountersManagerBase.GetCreationData<TManager>().ToArray());

    return installer;
}

Answer 1

Perf counters are based on shared memory segments. I have seen similar problems when someone still has a handle to the shared memory segment. This can include the perfmon tool itself. You may be able to use the handle utility to see who is holding onto the shared memory. You shouldn't need to reboot so try to see if this problem goes away of you shut down perfmon first and then re-create the counters. Also make sure that your service is not running.