CAN A LENS BE TOO FAST?
It's hard to see how it can be, at least if you can stop it down. After all, it's quite easy to stop even an f/1 or f/0.95 lens down to f/2.8. but it's impossible to open an f/2.8 lens even to f/2. A very fast lens may however be too expensive; or too bulky; or too heavy; or too vulnerable; or too obtrusive. It may deliver inadequate technical quality or inadequate depth of field; or be too hard to focus easily. It may flare too much. It may not be ergonomic. All of these features will very likely be related to its speed, but none of them is quite the same as "too fast" in any absolute sense: they are all shortcomings which may mean that a slower lens is a better bet, but which equally may be shortcomings that can perfectly well be traded for the advantages, when they are needed, of sheer speed. Let's look at drawback in turn.
An extra stop of speed, or even half a stop, can double the price of a lens. But the truth is that while f/1.4 or f/1.5 lenses are expensive at 50mm, plenty of photographers can either afford them, or at least find the money. Go to f/1.2 or f/1 and it's another story. The same is true of very fast lenses in other focal lengths: an 85mm or 90mm f/2 is what you might call "normal fast", but most people find the price of f/1.4 or f/1.2 eye-watering.
A kind friend once lent me a 50/1 Noctilux for a year. I didn't use it very much. Partly this was because it wasn't mine (see "cost", above) and I didn't want to break it or even put any marks of wear on it, but also, next to my 50/1.5 lenses (I had both a C-Sonnar and a Nokton at the time) it was huge, which made it inconvenient to carry. With any focal length, and with some specific designs, there comes a point where you decide that a lens is just too big.
A lot does depend on the design, though. I have a 135/2.3 Vivitar Series 1 in Nikon fit and a 135/1.8 Porst in Pentax fit. They're comparable in size, despite the Vivitar being half a stop faster, because the Vivitar is more solidly made and focuses ridiculously close.
Filter size can be relevant, too. I also have a 200/3 Vivitar Series 1 and a 200/4 Nikkor. The Nikkor take 52mm filters as against the Vivitar's 72mm, and while both have built-in hoods, the Nikkor's is much deeper and more effective. The Vivitar focuses closer, though.
This is closely related to bulk, obviously, but it's not quite the same. Some fast lenses are quite small: the much-prized last pre-aspheric version of the 35/1.4 Summilux is tiny, though quite heavy for its size. The current aspheric version is vastly better, but also vastly heavier. For the kind of pictures I take, I prize speed above image quality. Also, I prize not having a stiff neck from carrying a too-heavy camera. If anything (God forbid!) happened to my Summilux I'd go for a 35/1.7 Voigtländer as a replacement.
Having half an acre of glass on the front of your camera increases the risk of damage. Big, heavy lenses have more inertia, too: if they start swinging on a strap around your neck, they keep moving and they hit things harder. Of course you can put a filter in front of it, but this still further adds to the bulk, weight and expense.
A great big lens tends to attract more attention than a small one, though it's quite easy to overstate this one because even quite slow variable-aperture zooms can be pretty huge if the focal length range is long enough. Think, though, of a 300/5.6; a 300/4; a 300/2.8; and a 300/2 (Nikon made one).
6 Poor technical quality
It's easier to make a sharp, contrasty, low-distortion slow lens than a sharp, contrasty, low-distortion fast one. Compare the legendary Micro-Nikkor 55/2.8 with, well, anything faster that Nikon ever made in the 50-58mm range. At f/3.5 even a 4-glass Tessar-type lens can be very good, especially if you stop down to f/5.6 or less. As you go faster, you need more glasses and cleverer designs.
This problem was much greater in the past than it is today, when designs were "stretched" to get more speed at a given price, or sometimes, at any price: quite often, over-stretched, like the 50/0.95 Canon rangefinder lens, which wasn't half as good as its f/1.2 cousin. Then again, the f/1.2 was pretty mediocre next to the contemporary Canon f/1.4.
Sure, you can make a sharp, contrasty, low-distortion fast lens: the Zeiss 55/1.4 Otus is staggeringly good. So it should be. There are 12 glasses in 10 groups; it weighs about a kilo (over 2 lb); and at the time of writing it cost just under $4000 in the USA or £2669 in the UK. It also takes a 77mm filter. For that matter a 50/1.4 Nikkor is very good lens; and again, so it should be for rather more than twice the price of the 50/1.8, only 2/3 stop slower. At that, it's still under a tenth of the price of the Otus.
Again, go outside the (relatively easy) 50-55mm focal length, and the problems are all exacerbated. Even at 50-58mm, though, vignetting is often a problem.
7 Shallow depth of field
You can use this creatively: this used to be called "selective focus". You can compose your pictures so it doesn't matter: this was the traditional way to use ultra-fast lenses. Or you can do it just because you can, to show that you're a clever photographer who owns a fast lens. The third approach normally shows clearly that you own a fast lens, but shows equally clearly that you are not, in fact, very clever.
8 Difficulty of focusing
This derives directly from the above. Shooting at full aperture with a fast lens means that you need to focus very carefully, especially close up: for example, on the eyes (or on the nearer eye) in a portrait. Focusing to this degree of accuracy can be difficult if you do it manually. Rely on autofocus, and it can be even more difficult if your point of focus requires playing around with focus points on the screen.
Bigger front elements (and bigger internal elements, for that matter) mean bigger areas of glass to catch light sources or simply to increase veiling flare. They also mean that you need bigger, deeper lens hoods. These can be bulky, heavy and more or less inconvenient, or they can be skimpy, sometimes to the point of uselessness.
In theory, a lens can be too small to handle, but equally, you can always add a focusing spur or a focusing collar to a small lens. With a big one, you're stuck with big diameters. This may or may not matter with shorter focal lengths, but long ones can get very unwieldy, quite quickly.
THE DIGITAL GAME CHANGER
Until comparatively recently, 35mm films were surprisingly slow. There were always outliers, such as the short-lived, grainy and soft Ilford HPS which was introduced in the mid-1950s as 400ASA then sold as 800 ASA after the 1960 speed revision, but mostly, 400 was top whack for B+W until Kodak TMZ appeared in about 1980 with a base speed of ISO 1000 or so and a very long toe that made it ideal for pushing. Ilford's Delta 3200, which appeared rather later, was about 1/3 stop faster. Although a few seriously fast colour films appeared, anything much over ISO 400 was unusual and sufficiently expensive that they were never popular: such exotica as Agfachrome 1000, Ektachrome P800/1600 and Konica 3200 were short lived. Even at EI 3200, fast lenses were often extremely desirable.
Then, in the early 21st century, digital ISO speeds first caught up with film, then surpassed it. The Nikon D4 in February 2012 offered quite extraordinary speeds, with remarkably good quality even at ISO 12,800, though the "push" speeds were rather less satisfactory. The same sensor is still used in the Nikon Df .
This meant that ultra-fast lenses were seldom needed just to get enough light on the sensor, and their whole raison d'etre began to change. They were bought (and sold) for their "look", especially for ultra-shallow depth of field but also for vignetting, soft edges and all the other things hitherto widely regarded as "faults". Although for decades my standard lenses on film were 35/1.4 and 50/1.4 (and even f/1.2, for a while), my standard lens now on the Df is an f/2.8 Micro Nikkor. After all, ISO 6400 with f/2.8 allows the same shutter speeds as ISO 3200 at f/2, ISO 1600 at f/1.4, and ISO 800 at f/1.
Although I sometimes use selective focus, I find that most of the time, the f/2.8 of the Micro Nikkor is more than adequate. Then again, I really dislike shallow depth of field in pictures where it's totally inappropriate; where someone is just using it because they can.