I ran across a few rocky areas on the interior of Oak Island Nova Scotia, but I still used a large search coil instead of a small search coil.
Normally a small search coil would be a better choice for metal detecting in and around rocky areas.
The large 17 inch search coil on my CTX 3030 performed really well, on both large and small size targets.
Sweeping my 17 inch search coil just above the top of the rocks, I recovered two Indian head pennies in a craggy inland area.
I figured just above the rocks, my 17 inch search coil was 7 or 8 inches above the surface of the ground and I would still detect targets a good 9 or 10 inches below the surface.
These two Indian head cents are dated 1905 and 1908, they both came out of the same hole between two rocks.
When you are metal detecting over rocks, it is wise to consider the depth advantage of a really big search coil.
A 6 inch search coil on my CTX 3030 would have been easier to move around the craggy rocks, but I actually got better target depth sweeping the large 17 inch search coil over the tops of the rocks.
I also covered the craggy ground a little faster, not having to go around every nook and cranny in the rocks.
The size of the rocks in a rocky area will dictate what size search coil is best to use.
A couple of years ago I reverted back to using a hand held pin pointer to help me recover targets in rocky areas.
I know from my beach hunting experience that the harder it is to recover a target in a rocky area, the less hunted that area will be.
Large search coils, pin pointers and screened metal detectors, can all help you to have success in difficult to detect rocky areas.