I saw somewhere Titleist saying with this year models all balls are within a yard of each other. I have played not balls side by side the v1 and the v1x and have not noticed any diferrence except strike based but similar swings will get the Same distance with a touch different flight and feel.
I was similar to you in always using the x but I have been focused on putting and chip shots this year and have gravitated towards the prov1 due to that soft feel on those shots and I have ended up liking it in the full shots as well.

There is still the best way to know by comparing them from green to tee. Adam Scott use the V and Jordan uses the X. So club head speed is not the sole determinant. Adam is faster than Jordan. I'm not like you at all, but for me the V gains distance over the X due to roll-out. Along with the feel around the green, I made the switch. The spin characteristics are so different now that you can't compare either to 2-3 generations ago.

I always carry a little cheat sheet as I call it, that a friend, Bob Andrews gave me, who is our local Titleist sales rep. It is a wallet size card with a bunch of good info on regarding the two balls.

Feel: V - softer X - soft - V is softer
Flight: V - mid X - high - X flies higher
Long game spin: V - low X - low - equal
Iron spin: V - less X - more - iron spin is more with the X ball
Short game spin: V - high X - higher - higher spin rate with the X ball

It also states that the V - longer distance and more consistent flight as the X - more consistent flight.

Maybe this comparison might help answer you question. The X ball appears to spin more and is not as long, even though the V is a softer ball.

I share this when these questions come up at the course, and they do rather often.

I have played the prior prov1 and loved it.Always loved the soft feel when putting and chipping .
But I just bought a dozen provx,balls.
Went out last weekend to play a round,the X played great,definitely noticed a higher ball flight but distance was great,putting and chipping were fine,the X did not seem clicky at all.
Finished my round with the same ball,still in good shape.
I guess the best way is a ball fitting to see which ball fits you the best,especially with a trackman to give you total shot comparisons.
Plan on picking up some Prov1,balls and doing some shot comprarison.on course.
Hope I can find a ball fitting in my area soon.

Personally, I would prefer a mix of the two. The flight from the Pro V1x and the soft feel and sound from the V1. I used to use the Pro V1, but has this year switched to the V1x since I figured the flight was most important since they behave the same at and are puns the green.