I figured it out. Instead of adding an appearance, I added just a surface finish. I hadn't noticed there's a "from file" in the selection menu. With that, I could add a sponge texture, without the color. Wow. Wish I'd learned this long ago.
That's great information. Thanks for sharing the discovery.