Fresh sounds in 2012
05.03.2012
New devices all around! It is autumn once again and there’s a lot happening on the market. New devices are out, featuring advanced functions – and they are all available at a similar, quite affordable price. But which one to choose? Because of this question we are launching our third large 2011/2012 test. We will introduce the competing innovations – and which device will please your ears the most!
The test candidates
Just as in our previous tests in 2009 and 2010, we have preliminary selected twelve devices from the large amount of test candidates which we had liked in previous tests. All devices had to prove themselves in three test scenarios. Very affordable devices, recent top-selling devices and freshly released devices were in the race. The following devices were tested (the link leads to the respective individual test reports): Olympus VN series, iPhone 4, Android (Samsung Galaxy Mini), Olympus DS-5000, Zoom H2n, Olympus LS-5, Zoom H4n, Olympus LS-3/7, Tascam DR-40, Olympus DM-650/620, Roland R-26 and Sony D-50. In one of the three tests, a Tascam DR-07MKII and a Zoom H1 were also assessed.
----------------------
Directly to:
----------------------
What – and how – we tested
Neither the appearance, nor the functions or the usability of the devices were evaluated in this series of tests. We have described these features in the individual tests already; furthermore, these aspects have to be assessed in the context of practical use, i.e. depending on your specific needs. As a musician you might urgently require an XLR connector and a four track recording; as a scientist you may favor a physically small device. In this test, the one and only important aspect is recording quality – in our case a combination of 1) voice quality / detail perception, 2) a low amount of noise and 3) the amount of spatial perception provided by the recordings.
In order to evaluate the devices, four testers listened to all recordings through an excellent headphone (AKG K701). A fifth colleague had previously anonymized all recordings (e.g. the recording from the Sony D-50 was called ‘‘seagull‘‘, the one from the Roland R-26 was called ‘‘tortoise” and so on). Therefore, none of the testers knew from which device a recording originated. Also, all recordings contained the same sequence of sound. The testers separately completed a questionnaire on how they perceived the sound. For example: ‘‘seagull: full tone, very precise details, low amount of noise, very clear, wide stereo recording – slightly located in the best third”
For each device, we discussed different aspects of our individual impressions in a team. Then, we once more anonymized the best four recordings, listened to them, and ranked them again. After this step we determined which device would be the winner concerning the three respective aspects of quality mentioned above. Based on these rankings we determined winners for different recording situations: an ambience winner (recommendation for recording quiet sources in a distinguishable way), an interview winner (recommendation for interview situations), and a music winner (recommendation for music recordings).
Only after this thorough assessment of sound quality we supplemented our test with additional aspects, such as the device’s usability, size, power consumption or price. The combination of all these factors determined our price/performance winner.
Our test recording
There are usually quite various needs and areas of use when it comes to digital recorders. You might like to record your piano concerto or a lecture at the university, you might want to record the ambiance in a forest or you might simply need good recordings in many different situations. Of course, it is not possible to test recorders in every situation imaginable. After a lengthy discussion we picked three scenarios which, in our opinion, represented a good range of potential situations of use: ambience-, music-, and interview recording/recording of speech.
If you would like to retrace our test results and assess the audio examples yourself, you should listen to the recordings through a (good) headphone. Notebook speakers or ordinary computer speakers do not deliver the essential fine details or important nuances. In order to listen to the recordings produced by a specific recording device, click the “play‘” button above the corresponding audio sample. Furthermore, you can directly download the corresponding MP3 file for your own tests by clicking on the recorder’s name.
----------------------
Test 1: Ambience (church bells from afar)
For our ambient recordings, we picked a location at the banks of the Lahn river, close to Marburg University’s dining hall. We chose a place that’s inviting for a comfortable break, and from where the town bells can be heard at noon. In particular, the echo of the bells is an interesting feature of this location. Constant background noise comes from the nearby B3 highway and from the many students on the busy bikeway (10 meters/ 33 feet away from our recording position) proceeding to the near dining hall. All this made for an ideal place to put the recording devices to the test. Please listen to the recording samples with a good headphone.
In order to differentiate and to rate the recordings after having listened to the first sample recording, we agreed on relevant and distinctive aspects of the recording:
- Traffic noise: How do you perceive the background noise from the road? Is it rather undistinguishable noise or can it be identified as specific traffic noise?
- Birds and ducks: Do you hear the chirp of a bird or a duck? Can you clearly detect their position?
- Voices: How clearly audible are the voices in the background?
- Bells: How many different bells can you hear? Do you hear the bells’ echo?
- Bicycles: Do you hear the quiet approach of a bicycle after you’ve heard the bicycle bell ringing? The bicycle’s subsequent acceleration can be heard even better. How good can you retrace the bicycle’s path from right to left?
- Overall impression: Which recording sounds natural and detailed to you?
-----------------------
Sample recordings ‘‘Ambience“ (in alphabetical order)
The recording cannot be accessed in full due to an SD card error; however, all relevant details can be heard.
Review "Ambience"
What an acoustically complex recording situation!. We were very happy with the setting we chose: It enabled us to clearly pinpoint the relevant, distinctive features of the different recorders – and it was easy to declare a clear test winner. But first things first:
The worst recording was clearly produced by the Olympus VN-7000- this was not a surprise since this device is made for taking notes. The Olympus DM-650 sounds significantly better, but unfortunately it is still not suitable for the recording situation. While the DM-650 is our winner when it comes to recording conversations, it does come second to last in producing ambience recordings. There are two reasons for this: the automatic level control (note to the “wobbly” sound volume in the recording), and the strongly emphasized bass. The internal Roland R-26 XY-microphone does not perform brilliantly – the recording sounds hollow, with interferences, and the details appear blurred. However, the stereo perception is good.
As ‘average’ we assess the Tascam DR-40 (audible ambient noise, slightly booming) and the Zoom H2n (details are audible, wide stereo perception; nevertheless we still miss a balance in the middle; altogether comparatively hollow sounding).
We were very impressed by the recording produced by the Olympus LS-3. However, due to its intense bass, it appears a somewhat “mighty”, and unfortunately the noise coming from the road is very energetic. Nevertheless the recording sounds extremely detailed and natural.
Each of our four raters was impressed by the recordings of the Zoom H4n, the Olympus LS-5 and the Sony PCM D-50. Despite the slight rustle in the beginning (due to a static electricity charge from our windbreaker), the Zoom 4Hn (ranked third) produces a clear and balanced recording. The stereo perception is wide and does not appear unnatural. The Olympus LS-5 (ranked second) captivates with subtle and vivid details. Voices can be perceived well, the bicycle bell sounds very clear; overall the recording provides a pleasant, bright composition.
Finally, there is the Sony D-50 (ranked first) – what an impressive recording! Only this recording contains almost no noise, and the highway can be clearly identified as such; the echo of the bells can be clearly recognized as its own sound; the voices are clear and the recording provides realistic stereo perception. You can even hear the creaks of the bicycle as it approaches. In terms of ambience recording, this is clearly the best recording. Therefore, the Sony PCM D-50 receives the award “2011/2012 ambience winner”.
(Buy)
-------------
Test 2: Interview
As always, we recorded a conference situation in our conference room (30 square meters/320 square feet; ceiling height 4 meters/13 feet). Three people sat directly at a table where the recording device was placed, one person typewrote 2 meters/7 feet away from the microphone and another person later entered the room through a door 5 meters / 16 feet away.
We included two smart phones and a dictaphone in this test in order to get a better impression of the full range of possible recording qualities (many people do use their smart phones for interview recordings). Thus, twelve recording devices were placed on the table. All recorders were set up with similar settings. As far as possible we recorded at average microphone sensitivity, as MP3 with 320 Kbit, automatic scaling, with all extra effects switched off. Adjustable microphones were set to a 90° angle. We configured all devices according to the settings advertised by the manufacturers (e.g. the H2n with 2CH-setting, the LS-3 and DM-650 with the third (bass-)microphone switched on etc.)
-----------------------
Recording samples "Interview"
(in alphabetical order)
Review "Interview"
In our analysis, three devices struck us as being unsuitable for this recording situation. They are unsuitable because the ambient noise is too intense, and because the overall comprehensibility is insufficient. The voices crackle like in a 40s movie. The content is understandable, though we realized that listening to such recordings is painful to the ears after a while. The three devices were the iPhone4, the Olympus DS-500 and the VN-7000. These devices are not recommended for recording talks including a bigger amount of people.
Another recorder can be recommended with limitations, since it has significant shortcomings in terms of recording quality. The voice whirrs, the sound is flat, though the recording is fairly comprehensible. This recording was recorded with the software Tape-a-Talk on the Android cell phone.
We found out that cheap recording technology – which can be found in iPhones, Android phones or in the cheap Olympus VN – produces recordings with clearly audible losses in terms of recording quality; these devices are merely emergency solutions if the need for a recording quickly arises.
All other recorders (eight in total) are doing a very good job in this recording situation. There are for sure audible differences in sound quality amongst these eight recording devices: for example concerning the bass level (e.g. in case of the LS-3 and DM-650 relatively more intense, in case of the Sony D-50 rather less intense), the stereo width (the Zoom H2n features slightly more width, the Roland R-26 a bit less); the echo (the LS-5 produces a bit more echo, the DR-40 somewhat less). In our opinion, none of these differences are relevant for recording conversations consisting of 2-10 persons, since all eight recorders produced convincing recordings. The conversations can be comprehended in a good and realistically sounding way. This makes the recordings suitable both for transcription and for potential publications. We suggest choosing one of these eight recorders for recording interviews – which one is chosen in particular, should be guided by your budget, and your personal preferences and taste.
Nevertheless, a surprising amount of devices delivers very good recordings of human language. As mentioned above, there are differences in tone quality/tone composition – but this is not vital in terms of good comprehensibility of the conversations. If you consider the price a determining factor, we recommend the Olympus DM-650 for interview situations consisting of 2-10 people. It is therefore our “2011/2012 interview winner”. (The DM-650 (Europe) and DM-620 (US) are the same recorders).
-------------
Test 3: Music
It is 7 PM on Monday, the 14th of November, 2011. The time has come: violins, violas, flutes, timpani, double basses and trumpets are bustling on stage, 50 student musicians put them into position and tune them to standard pitch “A”. We feel honored that we can record here – after all, the Marburg student symphony orchestra, conducted by Ulrich Manfred Metzger. There is no other audience than us and our recorders – we almost feel lost in Marburg University’s auditorium maximum, which is almost 1,000 square meters/ 10700 square feet big. However, the room provides an interesting acoustic setting, and it was recently refurbished – a promising and unique setting for our 2011 music comparison test.
We set up our 11 recording devices in the middle of the fifth row. In order to provide comparability, all devices were adjusted as similarly as possible (e.g. microphones were set to 90°, although some models would have sounded much better if the microphones had been set to 120° - which we would recommend). At the beginning of rehearsal we manually adjusted the devices’ input level – and then we simply enjoyed the sight of a practicing orchestra: A fascinating experience, as we listened to the students playing and the vivid comments of the very kind.
If you listen to the sample recordings, please beware that the differences must not always be synonymous with a worse or better quality. Every recording device has its own characteristics in terms of sound. It captures audio and transfers it in a slightly different way onto the SD card. Whether a good recording should emphasize the double bass or not is a question of personal preference. However, audible noise or flat sound can be seen as flaws.
We intentionally requested for a very early appointment, so we could be there during the practice of a new piece. During this phase of “unfinished” playing, off notes and off timing enabled us to distinguish instruments – and therefore the performance of the recording devices – in a better way. Our two-minute excerpt contains such key points in order to provide good potential for comparison. Here are some questions to guide you when you’re listening to the recordings: During the first seconds, you hear rapidly playing strings and flutes. Can you differentiate between the flutes and the strings, before the flutes stop playing one by one after approximately seven seconds? At around a 1:10 the piece slowly gets quieter; at this time you can hear humming noise in some recordings. Finally, the drum and double bass provide a reference point for a successful recording of bass. In a good recording it should be possible to clearly differentiate the instruments from each other, even when they are playing simultaneously (for instance at 0:38 or 1:05). For those with good ears: at around 0:34 you can hear a click. We assume it is the conductor’s baton hitting the music stand. While this click can be heard in every recording, it does not always sound quite crisp and clear. The representation of space differs significantly. Can you perceive the distance between the instruments? How closely together or how far apart from each other do the instruments sound? An authentic recording which represents our experience in the lecture hall should provide us with a clear sense of space – without the instruments sounding too far away.
----------------------
Sample recordings "Music"
An abstract from the overture ‘‘Coriolan“ in c minor, op.62 by Ludwig van Beethoven. (Ordered from best to worst recording)
Review of the recording situation “Music"
Our anonymized assessment of the recordings quickly showed: The majority of the recorders delivered really outstanding recordings. However, some recordings were strikingly different – both in a positive and a negative way. In terms of quality, three groups could be formed:
As we expected, the Olympus VN-7000 provided by far the worst recording – it is a simple note taking device. The Olympus DM-650 also failed – just as it did in the ambience test. It cannot handle the dynamic music due to its automatic level adjustments, and a there is quite an amount of noise in the recording. The recording sounds lost and it ‘’falls apart”, it whirs and hisses in quiet passages. The Zoom H1 also falls far behind in comparison to the other devices. We did expect more here. The recording sounds hollow and “far away”. One of our testers commented: “did someone place a cover on the microphone?”
The devices in the broad midfield produced consistently good recordings in this recording situation. The recordings do not have errors and they are clear. However, in direct comparison to our favorite devices, those recordings did not provide such sharp distinctions, and special perception felt a bit “narrow”. Bass and drums are not easily distinguishable in loud passages. The midfield consists of the Tascam DR-40, the Tascam DR-07MKII, the Zoom H4n and the Sony D-50 (we had anticipated a much better performance by the latter device). On the basis of our recordings, ranks 4 and 3 are awarded to the Olympus LS-3 and the LS-5.
To our surprise, the recordings produced by the Zoom H2n and the Roland R-26 were really convincing. These two devices provided us with precise recordings of the instruments, while recording with a very low amount of noise and in a balanced way. The Roland is razor-sharp and extremely precise. The Zoom H2n is more pleasing, sounds softer – though that’s a matter of taste. No matter what you might prefer: both recorders deserve to be our “2011 music winners”’.
Zoom H2n
Roland R-26
Some self criticism
We had a controversial discussion about the results. We were astonished by the fact that the Roland and the Zoom were that far ahead of the field, though they did not perform well in the ambience test. We are very happy for the winners, though we wonder whether the microphone setup co-determined the results. In order to equalize the test situation, we had set the microphones to a 90° recording angle. But clearly, a wide microphone angle is much more appropriate for recording an orchestra. We assume that, by creating a putative “equal” recording situation, we were did not do justice to some devices. In our test, the Zoom H2n recorded with Mid-Side (MS), the Sony D-50 in a 90° position, which makes the recordings sound more flat. The Roland used all of its 4 microphones together, which might have lead to a decisive advantage. This is not satisfying to us – at this point, we want to know what’s what. Unfortunately, this means more work: We are going to do a second concert recording in the coming months and we will set the microphones to wide/120°/MS wherever possible. If no change in the ranking occurs, the results are bullet-proof.
Edit, 18.01.2012: We have made an effort to repeat the test. All results (and they largely confirm our current test winners) can be found here (in german language):
http://www.audiotranskription.de/test-2011-update
-------------
Price/performance winner
We looked at several aspects while searching for a price/performance winner. On one hand, we obviously considered the test results and the recording quality. On the other hand, the price, the handling and the feedback from users were considered. The Olympus LS-5 made it into the upper third in all our tests; it is very well manufactured (better than most other devices), and so far we haven’t had any warranty cases. The LS-3 is a little bit weaker in terms of sound quality (it doesn’t provide fine highs; therefore, recordings sound not as vivid). The device is smaller and lighter and its batteries can be charged in the device – this saves money on a charger. We would recommend the LS-5 for music and ambience recording situations, and the LS-3 where language is the main focus. Having that said, both devices are, in our opinion, on an equal footing – this is why they both are our “2011/2012 price/performance winners”.
The test team
Transparent description of the researchers’ role and opinion is a vital part in scientific research. Only this openness enables the readers to comprehend from which point of view our assessments were made. Dr. Thorsten Dresing and Thorsten Pehl are education scientists. Our core skill is the recording of interview situations for the purpose of scientific analysis of conversations. For some years we recorded and transcribed interviews. Every day we are listening to recordings through a headphone. This where we come from: A better recording (i.e. a more comprehensible and realistic recording) makes it easier to subsequently transcribe the data – and it makes it easier to “get your mind into the recording situation” when analyzing the conversations. Our colleague David Georgi has worked as a journalist for a long time. He authors contributions on sports, travel and nature – those settings call for a special focus on the clarity of recordings of nature and music. Hence, David Georgi reviewed the recordings concerning their “natural” sound and clarity. We consciously did not consider the analysis of frequency curves or noise intervals in our tests. Wearing headphones, we decide with closed eyes whether a recording sounds good or bad. All our sample recordings can be downloaded here, so you can comprehend our evaluations at any given time. We hope this will help you finding your personal favorite!
Last but not least:
We earn our livelihood by selling recording devices. Our aim is to assess the products as transparently as possible - in order to offer well-grounded information and possibilities for decision-making beyond the promises of advertisement. If you would like to support our work, we would be pleased to welcome you as customers in our online shop.
For the ambient and music test was the LS-3/7's third bass mic turned on? If it was this could account for the difference between the high and low end differences.
Great reviews and audio samples, thanks a tone.
Post new comment