Okay, I've got a fix, but it's maybe not optimal yet.

My first thought was, "okay, ObjectProperties need to be released from the dictionary when I remove nodes from the scene. Let's make Node implement IDisposable, and then each child of Node can make sure that its properties get released when it disposes."
I did this for Node and its children. Didn't take too long. And when I profiled it, I did get some memory back when removing nodes. A lot of their ObjectProperties got collected.

But not all.

So I did a search to see everywhere else that Property was used in Balder and there are a lot of places. Materials, etc. I didn't want to have to implement logic to decide when to dispose all of those, so I looked for another option.

I ended up replacing Dictionary with a WeakDictionary. I made a dictionary that automatically cleans up the ObjectProperties. It removes them if the object they applied to is no longer alive.

The downside is that cleaning up is an expensive operation, so you don't want to do it too often. I ended up using a random decision and it performs a clean after adding a value 1/100,000th of the time. That percentage turned out to be small
enough not to noticeably impact my framerate (When I did it 1/1, I dropped from 60ftp to 18fps.) but still often enough that memory usage doesn't climb to 1GB just from adding and removing a single cylinder a few hundred times.

I still don't know that this is the best solution. Seems to work for now. If anyone is interested, I can post the code.

Thanks!