There are a couple of different ways in which rules require, restrict, encourage, or discourage certain kinds of technical content.
1) Short programs have always required exactly 7 or 8 (depending on the year) specific elements or kinds or elements with significant penalties for not meeting the requirements.
2) Free programs in Lynn's day had no explicit requirements or restrictions that I'm aware of. There were expectations of the different kinds of elements one would expect to find in a singles (or pairs) free skate, but no explicit penalties for doing too many or too few -- it was up to the judges to reflect the balance of the program as they perceived it.
By the 1980s there started to be restrictions on repeating triple jumps (Zayak rule) and then requirements and limits on the numbers of jump combinations and/or sequences in the free skate.
In the mid-1990s, the ISU introduced well-balanced program guidelines to encourage all skaters to include a minimum number of spins and at least one step sequence in their free program. By about 2000 those guidelines became requirements -- e.g., at least one jump combo but no more than three, at least four spins, one step sequence, and one spiral sequence (ladies) or field moves sequence (men) -- with deductions for omitting requirements (or repeating too many jumps or doing too many combos).
Look at Plushenko's free program at 2000 Worlds for an example of what happened when the skater left out spins to try to fit in more jumps. I think 2000 NHK, where Slutskaya with a couple of visible errors beat a "clean" performance by Butyrskaya, is another example. Butyrskaya did do steps and spirals during her program, but not really in what could be considered official sequences (especially step sequence), so she might have been penalized for lacking a "required" element in the free program.
I haven't kept track of all the changes for pairs over the years, but as of the mid-1990s the guidelines for pairs freeskate didn't leave much room for variation because there are so many different kinds of pair elements that the guidelines recommended and then required something like "at least one but no more than two" (death spirals, throw jumps, twist lifts, etc.)
So that's where things stood at the end of the 6.0 era -- the free program was much more homogeneous in the number of each kind of element than it had been for most of the 20th century, and there were some very explicit requirements that would be penalized if omitted, but there was also still room for skaters to emphasize their strengths and deemphasize their weaknesses.
Then the IJS took over. And instead of just requiring a minimum number of each kind of element, instead the new well-balanced rules imposed maximums for each kind. But if you want to earn as many points as your competitors, you'd better fill all the element slots allowed, because doing fewer elements gives you fewer opportunities to earn TES points. So if the minimums and maximums for each kind are effectively identical, then every skater will have the same number of each kind of element, unless they make mistakes or choose to leave something out or choose to include an extra element at the end they know they'll get no points for. Thus the similarity in program content across the entire field.
The exact numbers of elements have changed over the last 8 years (one spin has been deleted, and one of the sequences has changed from leveled to "choreographed").
In addition, the new well-balanced rules also required specific elements within the general categories. In addition to the preexisting rules about jumps and jump combos, an axel jump (single, double, or triple) was now required in the free skate. Of the three or four allowed spins, one must be a combo spin, one must include only one basic position, and one must have a flying entry, and also no two spins can have the same IJS code. That rule is designed to ensure that all skaters showcase certain basic spinning skills, but especially with only three spins available now, each of which has to fill one of those requirements while maximizing potential points, there's no opportunity for skaters to do a different kind of spin that would fill one of the already-filled requirements in order to showcase their own unique skills or enhance the choreography at the expense of technical points.
3) The Scale of Values lays out exactly how much each specific element is worth in both base mark and grades of execution. This allows skaters to strategize their program content to earn the best theoretical base mark, or to aim for the best actual technical score they personally are capable of, taking likely GOEs into account. That's a big advantage of the IJS.
However, in many cases, especially for jumps but also for higher vs. lower levels in other elements, the differences in GOEs vs. base marks are such that it's clearly more valuable on paper to do a harder element with small flaws (or, right now, with major flaws on rotated quads and triple axels) than to do an easier element exceptionally well. So just looking at the SoV skaters and coaches would often choose to add difficulty before perfecting quality.
Also, if you clearly rotate a jump or clearly execute the right combination of features on a non-jump element, you will get credit for the element in the base mark. The technical panel calls are not matters of personal opinion (yes, sometimes differences in human perception play a part in the so-called gray areas if the execution of rotation or feature was borderline), whereas the GOEs from judges are more subjective and variable, and initially international judges were overly hesitant to award the highest positive GOEs. So it became even more true that doing a harder element just barely adequately was often more valuable than doing an easier element very well.
Hence the tendency of skaters to aim for doing level 3 or 4 spins and steps (and spiral sequences when they were leveled) if they had any hope of getting credit for the features, because they might know in advance or have reason to expect that doing an easier element with better quality would not earn them as many points.
And so complicated spins and step sequences became the norm and simple ones done well became rare.
4) The way the IJS defines and scores jump sequences, combined with the limits on total number of jump elements, means that some ways that skaters used to link jumps together as a kind of extra difficulty now continue to add to the risk of mistakes without adding points and in some cases subtracting points from the base value (e.g., 0.8 sequence multiplier, or using two jump slots for two jumps very close together that would have been considered a single sequence in the old system). So those kinds of jump series are no longer worth doing.
5) The way the levels are defined for various elements require or encourage most skaters to include the same features if they want to earn a higher level.
Anything that used to enhance and give extra technical difficulty to a spin or step sequence but that does not qualify as a feature is no longer an asset to the base value. If it's also likely to result in lower GOE, or at least not add to the GOE or not significantly, there's no reason to do it. For example, holding the free leg in a well-turned-out attitude position in a layback spin will earn fewer points than holding a position that also counts as a feature, even if it's less attractive. So the skaters with enough flexibility to enhance their laybacks in multiple ways will choose the enhancements that give them higher levels even if other choices might be more beautiful.
Of the various enhancements that do add to the levels, some are just a little more difficult than doing the basic element well, some moderately difficult, and some much more difficult. Therefore the vast majority of skaters will choose the easier features and few will choose features that are too hard to get credit for. (With some exceptions. E.g., Rachael Flatt and Joannie Rochette don't have the body type to do as many flexibility moves in their spins as some of their competitors, so they chose to develop the skills to make use of the opposite-direction spinning feature that most skaters don't bother with.)
Also, there's no value to doing just one feature in a spin or sequence, since the minimum to earn a higher level is 2 features for level 2. That leaves out some of the examples of basic spins with one enhancement that we used to enjoy under 6.0. And the "variety of steps and turns" feature is mandatory for any step sequence aiming at level 2 or higher, so there's no reason to cultivate skills to do step sequences that consist only or primarily of steps (and little jumps) or only or primarily of turns, or the same kinds of turns in different directions.
Well, now the senior men have a choreo step sequence in which they can emphasize other kinds of step skills. But that doesn't help the juniors or the ladies.
Some of these reasons for similarity of content between skaters are inevitable results of any attempt to tell skaters what's expected of all of them (what constitutes a well-balanced program). That's true under either judging system; if today's well-balanced program rules were used with 6.0 judging, we'd see the same similarity in program content.
Personally, I don't see the need to be so exact about exactly 7/8 jump passes, exactly 3 spins, etc., in free programs. I think there could be ways to rewrite the well-balanced rules to give more flexibility and still keep the optional elements comparable to each other according to each skater's choice. But so far the ISU hasn't seen it my way.
Some of the similarities are due to de facto requirements, not de jure. Skaters are free to choose other elements and other features, but if the elements they prefer to do are worth fewer points or are less likely to be credited, they have reason to leave them out and do the same ones that everybody else does.
Are there ways that the well-balanced rules, the GOE values, and/or the level definitions can be rewritten to give skaters more freedom in choosing to do easier elements with hope of reward in GOE and/or PCS or in choosing less common features with expectation of sufficient reward?
Going back to a situation with no restrictions at all is probably not the way to go. There are reasons those restrictions were put in place to begin with. Take them away, and skaters will again find ways to exploit that freedom the same way that they now exploit the restrictions. But I think there should be more of an effort to design the restrictions to give structure and equivalencies to variations between skaters, rather than to try to homogenize everyone into demonstrating the exact same sets of skills.