"Sauce for the goose, Mr. Savik, the odds will be even." Mr. Spock was wrong. The odds were not even. Entering the nebula gave Captain Kirk an advantage. When you raise the level of difficulty, you make the difference between the average and the expert more pronounced.
Always love a Star Trek reference and your right. Higher difficulty does make the difference far more pronounced between casual and master gamer. It is one of the reasons I tell people a game is not beaten until you can finish on the hardest settings.
I don't agree that higher risk=higher reward is always necessary. Lets face it, many of us do remarkably difficult things just so we can have a little achievement next to our steam account that says we did it (How many people here actually did spend weeks destroying all 20,000 Grox planets in Spore?). Even then, many times do we self impose our own restraints on games to make them harder (Beat Final Fantasy 1 with only 1 White Mage. That's was a favorite of mine). Even when not looking for bragging rights, we often do it to make the game last longer and feel fresh again and again.
To that last part about locking in the difficulty, I am mixed. Personally, I feel that if you change difficulty to a easier setting mid-game it amounts to cheating. That said, I also don't think that a player should be punished because 5 hours in they realized they bit off more then they can chew and must start over to correct the problem. It would really depend on the type of game itself. In games where difficulty affects mostly the environment around you and not the narrative (Like Star Control, Wing Commander, Mass Effect and Fallout for ex.) I can see the use of a shift able difficulty setting. In games where difficulty is key to how the narrative can play out (More RPG/Multi Ending games then I can count do this) then the concept of a changeable difficulty causes problems and is at best, difficult to include.
I think what it comes down to, is difficulty just a modifier or does it affect how the story can play out? If Star Control is to be an adventure game, then I can see both as possibilities, depending on the writing.