Related
Hi All,
I am trying to test some image analysis applications with the Huawei P9. Is it possible to extract two images (one from each camera) from a single shot? I know one of the cameras has a monochrome lens, and I know how to obtain just the monochrome image, but it would be extremely valuable if I could obtain both images from just one shot.
Looking forward to your assistance,
Josh
I do not want to stop your enthusiasm, but from my tests, they don't exists two images from one shot.
I didn't do my tests with an engineering approach, I only did some empirical test and from these I gather that:
- when you setup the Monochrome mode, the P9 activates the left camera (on the left when facing the phone back)
- with all the other modes, the P9 activates the right camera (the one between the flash and the left camera)
The P9 doesn't create 2 images, than combine them, it just shot always 1. How I came to this conclusion? You can also try it at home:
I choosen few static subject and I made my photos with the phone on tripod, than I did many photoshoots in the normal way and also by covering alternately the 2 cameras with a black scotch tape.
Even by naked eye, even by using an image comparation software (I used Beyond Compare from Scooter Software) I found no difference at all, no more brightness, no more contrast, no better image definition.
I did in a bright environment, in a dark one, I enabled and disabled the PRO mode and I tried to do a testing more complete as I could (honestly, I omitted to test the image in RAW mode, I tested only JPEGs), but my conclusion is that the 2 cameras are doing a different job, but they are definetely NOT working together.
Thanks for testing, but did you also try this outside on a landscape view? Maybe then we will see other results?
Otherwise this is yet ANOTHER thing Huawei lied about.
Yes, I did.
I'm thinking about making a full post about photo comparation. Let's see
ScareIT said:
Yes, I did.
I'm thinking about making a full post about photo comparation. Let's see
Click to expand...
Click to collapse
That would be nice!
Hey guys. I did a quick test shooting in bokeh mode or aperture effect (I guess you know what I mean). If you cover the black and white lense it lets you shoot the picture BUT NOT edit the depth of field once you took the picture.
If you uncover the lense, it works like it is supposed and also stores the depth information (two lenses are crucial to get depth information).
Thus, in order to extract two images from one shot, the best guess is that you try it in bokeh mode. But even then I dont know if its possible. However, the phone definitely uses both lenses that time.
Great oTToToTenTanz!
I confirm that! Both cameras are essential to enable the wide aperture effect: when you try to shoot in the bokeh mode it appear an alert to check if the lens is clear, the blurred effect disappears and it's impossible to edit the depth in post-production.
I make 2 hypothesis:
- the phone really combines the 2 pictures in order to recreate the depth (is a strategy used in all the 3D cameras), so in some way there should be the possibility to get both pictures
- the phone uses the laser pointer to shot IR around the subject, then the monochrome camera will get the infrared information (and considering that its lens is without the RGB filter, will be very efficient to do that) and store them in order to obtain an accurate depth (I mean something like this: https://www.youtube.com/watch?v=dgrMVp7fMIE)
Nice things to try!
Additional Info on Depth
oTToToTenTanz said:
Hey guys. I did a quick test shooting in bokeh mode or aperture effect (I guess you know what I mean). If you cover the black and white lense it lets you shoot the picture BUT NOT edit the depth of field once you took the picture.
If you uncover the lense, it works like it is supposed and also stores the depth information (two lenses are crucial to get depth information).
Thus, in order to extract two images from one shot, the best guess is that you try it in bokeh mode. But even then I dont know if its possible. However, the phone definitely uses both lenses that time.
Click to expand...
Click to collapse
Hey oTToToTenTanz,
Really appreciate your (and everyone else's) help on this! Can you give me some more info on how you actually extract the depth info in a usable form e.g. a matrix? Does the image just produce an RGB-D image once saved?
Thanks so much,
Josh
Yes unfortunately I think this is simply a feature that huawei lied about. The phone doesn't actually use both lenses at the same time to produce better quality normal photos; the monochrome lens is only used for bw mode or to obtain depth information for the wife aperture mode. The two lenses are not used in conjunction to provide better low light performance. You can try it yourself as stated earlier in the thread, cover the bw lens with your finger and compare the photos with normal ones: they'll look the same...
As far as I understand it, there are two cases in which both cameras are used.
One is for the wide-aperture ("bokeh") mode, in which a depth map is created from both pictures that have a slightly different perspective. I've read somewhere that the resulting image is a normal JPG file that is way too large, so it seems that there is additional data after the end of the actual JPG image. This would also explain why the capability to adjust depth of field is lost once the file is opened and saved by any application. I'll have a look at such a file when I have some spare time; maybe I'll find out more.
The other case is landscape shots in low light. Several people reported that covering the second camera in this scenario results in much darker images. This seems like a silly limitation, but I believe I understand why it's there. The two images that the cameras take differ in perspective (obviously, due to the fact that the cameras are mounted next to each other), which is quite difficult to adjust for when trying to combine both sensors' data. However, when focusing at infinity, for example when taking landscape shots, the difference in perspective is negligible, so that in this case the two sensors' data can be easily combined to improve low-light performance.
Maybe it would be possible to combine both sensors' output at closer distances in a satisfactory way, but it seems that Huawei chose not to implement that. If I find a way to extract the second sensor's data from a wide-aperture image, I'll poke around a bit to see if it would be possible to combine them.
I did some poking around on my lunch break. I threw a wide-aperture image into JPEGsnoop and it came up with two images in the file (four if you count the thumbnails, as well), the first one being the processed, "bokeh" image, while the second is the original color image without any processing. I assume that this is the image that is used to re-process the wide-aperture image when editing the focus point or aperture through the gallery app.
JPEGsnoop also told me that there's more data after the image segments. Since it couldn't work out what that data is for (this is past the end of the actual JFIF file), I checked it out using a hex editor. I found a marker "edof" (extended depth-of-field?) followed by what looks like some header data, followed lots of repeating bytes. This block is about 1/16 the size of the image in pixels (so 1 byte for each 4x4 pixel block). I'm not sure whether that's a small greyscale version of the image itself or a depth map, but I suspect it's the latter.
So, I'm afraid that it will be impossible to extract the monochrome image sensor data from a wide-aperture image, as it's not there anymore.
PerpulaX said:
I've read somewhere that the resulting image is a normal JPG file that is way too large, so it seems that there is additional data after the end of the actual JPG image. This would also explain why the capability to adjust depth of field is lost once the file is opened and saved by any application. I'll have a look at such a file when I have some spare time; maybe I'll find out more.
Click to expand...
Click to collapse
I confirm that: I did few shots on a single subject (always using tripod);
- the pictures in normal mode and with wide aperture with the BW camera covered results in 2.5 MB weight (max resolution; the photo's Title/Subject/Description is marked as "edh"
- the same subject in wide aperture mode (with the BW camera fully working) results in 5.5 MB weight (more than double); the photo's Title/Subject/Description is marked as "edf"; if this photo is opened with some image editing software, no alpha layers or other visual information appears anywhere; if the photo is saved back, the size will became comparable to the same photo without wide aperture effect
As depth information are not appearing in any editing software, I suppose they are hidden inside the jpeg file with some kind of steganography technique. I tried to examine the file with some ready-to-use tool (like stegdetect, that should be capable to detect if a jpeg file is standard or has something hidden) but I get only some mismatching header error, nothing that can let me understand where and how the depth information are stored and, primarily, if the black and white picture is also stored inside.
The cam seems to be making two Images for every shot. You can for - instance - make a picture and then edit it with the onboard effects. If i make the picture e.g. partially B&W, I can see, that it does use an original B&W picture taken with the original shot. This is not an artificial B&W.
The question is, where it is stored or are the necessary informations only "combined"?
PerpulaX, ScareIT you guys are right,
- the 992x744 depth map is coded on 8 bits at the end of the file, use HxD editor to extract the image (check the tags in ascii code "edof" & "DepthEn" ).
- displayed jpg is the saved one after blur processing on your sd card
- hidden jpeg in exif is the original image shot , without blur processing.
So it explains why you can re-edit your picture anytime on your P9 even after renaming... or simply have fun with the depth map for detouring in photoshop for instance
Made a python script to automate the EDOF and image extraction. It's simple but it works.
https://github.com/jpbarraca/dual-camera-edof
zoubla88 said:
PerpulaX, ScareIT you guys are right,
- the 992x744 depth map is coded on 8 bits at the end of the file, use HxD editor to extract the image (check the tags in ascii code "edof" & "DepthEn" ).
- displayed jpg is the saved one after blur processing on your sd card
- hidden jpeg in exif is the original image shot , without blur processing.
So it explains why you can re-edit your picture anytime on your P9 even after renaming... or simply have fun with the depth map for detouring in photoshop for instance
Click to expand...
Click to collapse
Can you explain what is possible to do in post-process? What can I do with the photo?
You can do exactly the same thing as the Huawei gallery app (at least).
For Photoshop there are plenty of tutorials using Depth Maps with the Lens Blur plugin
ScareIT said:
Yes, I did.
I'm thinking about making a full post about photo comparation. Let's see
Click to expand...
Click to collapse
Waiting for more details and experience sharing from you
Tijauna said:
Yes unfortunately I think this is simply a feature that huawei lied about. The phone doesn't actually use both lenses at the same time to produce better quality normal photos; the monochrome lens is only used for bw mode or to obtain depth information for the wife aperture mode. The two lenses are not used in conjunction to provide better low light performance. You can try it yourself as stated earlier in the thread, cover the bw lens with your finger and compare the photos with normal ones: they'll look the same...
Click to expand...
Click to collapse
Hy!
I think, that P9 does take two pictures and combines them in low light conditions. Here is two example, when something went wrong with the combination of images, and the two images becomes visible: https://goo.gl/photos/cK5q2TEisEU7rmpz9
What do you think?
Abel
So the file size is increased when B&W is uncovered but gives no actual benefit to the picture? Damn it, as useless as Interpolation!
Hi!
After a HUGE amount of work, it's finally here!
Raw Dumper is a simple camera app for taking RAW pictures on the Zenfone 2 (ZE551ML)
(Intel-based smartphones rocks!)
Q: What's a RAW picture?
A: Jpeg pictures are processed, meaning that multiple image processing algorithms (debayer, white balance, tonemap, sharpening, saturation, noise reduction, jpeg lossy compressing, etc) were applied; they usually retain about 1/10 of their original data.
A raw picture, however, is taken directly from the camera sensor and all captured data is preserved, leaving to the user the decision of how the image is going to look.
Since they retain all data, raw pictures occupy more storage space (A single raw 13-megapixel photo use 23 MB, on the Zenfone 2 ZE551ML).
Q: How is this possible?
A: There is a hidden camera parameter on Intel-based phones that does the trick. It's mentioned in a system library called Intel Camera Extensions (that library also enables other camera features). However, the raw feature doesn't need the library to work.
Q: The pictures captured with this app are really RAW?
A: Yes, because the captured data has the Bayer pattern, the right size and has a lot of lens shading around the image's borders (the app tries to remove the lens shading, so you probably won't appear in your photos).
Some important notes:
I'm not responsible for bricked phones, broken phones/cameras/images or thermonuclear war
It is REALLY experimental
It's currently being tested on Zenfone 2 (ZE551ML) - but it should work on ZE550ML too
This app WILL NOT work on phones with non-Intel processors (Qualcomm, Samsung...)
It should work on Marshmallow stock rom
For working on custom roms that uses LL camera blobs (like Lineage OS), is required to patch the camera lib
It REQUIRES root (otherwise the camera subsystem will drain your internal storage by making another copy of every raw picture taken it in a root-only system directory; currently, there's no workaround to fix this )
The DNG pictures will be saved in /sdcard/DCIM/RawDumper (they can be opened using Snapseed, Darktable, RawTherapee and Adobe Lightroom)
It should work on the Razr I and Zenfone 5 too, but I haven't tested them properly because I don't own these devices
It only works with the Zenfone Zoom's front camera (the back camera requires the development of a specific custom kernel)
Screenshot:
Click HERE
Download APK:
Click HERE
Latest version: 3.5.0
Github repo:
Click HERE
Patched camera blobs for Lineage OS users:
(replace the /system/lib/hw/camera.vendor.mofd_v1.so file)
Click HERE
Have fun!
Created: 31/05/2017
Last update: 03/01/2019
To do list:
Low light mode
Picture format selection
Slow Sync Flash
And maaaaany other things...
Changelog:
Version 3.5.0:
Optimized APK size: from 6MB to only 2.2MB!
Better configurations switches's colors
Ultra fast picture saving thanks to asynchronous IO
Display camera aperture below shutter speed value
Added a new free-lens-camera-like effect
Fixed Original Size DNG tags
Fixed a minor interface bug
Version 3.4.0:
Optimized memory usage when saving raw pictures
Faster DNG compression using multiple processor cores
Less processor usage and garbage-collection-related FPS drops due to logcat background processing
Optimized update interval of the exposure metering features
Migrate our project to AndroidX
Added a switch to disable raw digest generation
Flash focus (beta)
Fixed the dates-related tags in Exif's data
Version 3.3.0:
Started using Adobe's DNG SDK to save the pictures
The raw files now can be losslessly compressed
The raw files are now are compatible with almost every RAW editor
Picture size selection enabled
The alternative color profiles are now saved on every raw file
Increased front camera screen flash brightness a bit
Version 3.2.1:
Several bug fixes (su shells, UI sliders, camera opening...)
Advanced Black Level Calibration
Simple tone curve
Camera lens vignetting toggle
Invert front camera pics toggle
New alternative color profile
Version 3.0.0:
Different white balance modes + manual white balance
The app's UI now auto-rotates
Fixed touch focus touch position
Added touch exposure metering
Reduced GPU overdraw
New logo! Made by Nazmul Hasan
Screen flash for the front camera
Advanced Lens Shading Correction implemented
Take pictures with volume down button/headset button/camera button (if present)
Focus with half-way pressed camera button (if present)
Fixed small graphics glitch when starting the camera
Fixed the shutter speed selector
Re-done the front picture mirroring due to incompatibilities with raw converters
Recalibrated the Zenfone 2 cameras under the Standard A illuminant
Version 2.3.1: (Quickfix version)
Fixed a not-so-rare race condition bug that crashes the app during its initialization
Fixed the shadows of certain buttons of the main UI
Removed some unused layouts from the main UI
Version 2.3.0:
Fixed a small glitch in the lower portion of the screen when the app is starting
The raw pictures now have the right rotation applied
Pictures taken with the front camera are also properly mirrored
ISO and shutter speed metering for the Zen2/Zoom front camera!
Fixed a weird bug in the lens shading correction algo
New gain based auto white balance algorithm that *should* always work
Version 2.2.1:
Focus peaking when using manual focus for a Mirrorless-like experience
A little bit faster picture taking on the Zenfone Zoom front camera
Version 2.2.0:
Added proper focus mode selection
Added manual focus (for compatible devices)
New animation when taking pictures
Added Zenfone Zoom to the compatibility list (front camera only)
A more detailed description HERE
Version 2.0.0:
A revamped inner architecture and user interface (with animations)
Added manual EV feature
Added ISO and shutter speed metering
Added a warning when the user is using Lineage OS with buggy camera blobs
Added Zenfone 5 and Razr I to the compatibility list (even though they need more testing)
The app (tries to) auto rotate the pictures (but it still quite buggy)
Added visual feedback when using touch focus
The size of the native tiff library was vastly decreased
Better white balance strategy for zenfone 2's front camera
A more detailed description HERE
Version 1.3.0: (may contain bugs)
Taking and saving pictures are executed in the background! YAY! (this may lead to hard-to-find bugs, however)
Taking pictures with the front camera is FINALLY HERE! (the front camera should work with custom roms using Lollipop's camera blobs! )
Some progress on taking raw pictures with the Razr I was done (but it won't work for now)
Version 1.2.3: (Quickfix version)
Better handling of alert dialogs
Force disabling of device list
Version 1.2.2: (Quickfix version)
Better log with some useful debug information
List of supported devices is temporarily disabled
Version 1.2.0:
(beta) Manual ISO: from 50 all the way up to 1600!
(beta) Manual Shutter Speed: short exposures like 1/10000s and long exposures like 32s!
Version 1.1.0:
DNG creation process updated with a WAY BETTER code! (Including better exif data!)
Now the app contains a list of supported devices and supported features (currently only ZE551ML and ZE550ML are supported)
Touch focus!
Changed some white balance calculations (need your feedback on this!)
Version 1.0.1:
The apk now contains only x86 and x86_64 compiled libraries.
Fixed a small bug when the flash button didin't update.
Now it should work on non-stock roms. Edit: it installs but crashes.
Minor updates on the DNG generating process.
Version 1.0.0:
First app release
Wasn't Asus Zenfone 2 unable to capture RAW images due to the restriction from sensor of the camera itself?
Testing, let's see how it works!
Also I've downloaded your project from github, I'm so curious about the structure.
SleepingAran said:
Wasn't Asus Zenfone 2 unable to capture RAW images due to the restriction from sensor of the camera itself?
Click to expand...
Click to collapse
Every picture you take using a digital camera was raw in some point of its processing. Of course, some camera sensors do all the processing inside and only dump the ready-made JPEG (this is not the case of the Zenfone 2). But, in theory all sensors (dumb VGA webcams, point-and-shoot cameras, DSLR professional cameras, smartphone cameras) could be capable shooting in RAW. In smartphones, most of the time, the limitation is on the camera software, not in the hardware. The Zenfone's (and other Intel) raw shooting ability is just a little bit hidden
pedperpat said:
Testing, let's see how it works!
Also I've downloaded your project from github, I'm so curious about the structure.
Click to expand...
Click to collapse
It's quite simple, is just a simple camera app with lots of hardcoded stuff (I'm going to fix it, don't worry hahaha). I just wanted to publish something to show is possible to shoot in RAW on the Zenf2. I'm planning to clean up the code and add some nice manual/auto ISO and shutter speed in the next versions (I've tested and these features work perfectly with RAW )
dreckneck said:
It's quite simple, is just a simple camera app with lots of hardcoded stuff (I'm going to fix it, don't worry hahaha). I just wanted to publish something to show is possible to shoot in RAW on the Zenf2. I'm planning to clean up the code and add some nice manual/auto ISO and shutter speed in the next versions (I've tested and these features work perfectly with RAW )
Click to expand...
Click to collapse
I had some problems to use the C++ libraries on Android Studio. I can't install the NDK, it's weird...
Btw, great job enabling the raw shoot mode at this phone.
pedperpat said:
I had some problems to use the C++ libraries on Android Studio. I can't install the NDK, it's weird...
Btw, great job enabling the raw shoot mode at this phone.
Click to expand...
Click to collapse
Thanks! I've updated the project to use the latest gradle plugin now, try updating Android Studio. NDK building is WAAAAAAY easier on the latest versions. If you need help, you can PM me
PS: I hate Android Studio hahaha
not working it's not compatible on my z00d mokee rom nightly 7.1.2, why?
dreckneck said:
Thanks! I've updated the project to use the latest gradle plugin now, try updating Android Studio. NDK building is WAAAAAAY easier on the latest versions. If you need help, you can PM me
PS: I hate Android Studio hahaha
Click to expand...
Click to collapse
I hate Android Studio at the update of NDK Libraries only. Android Studio is a great IDE despite of its slowly load times.
I'm at work, will try again your app when I came home
Let's check the github repo a little
dreckneck said:
Every picture you take using a digital camera was raw in some point of its processing. Of course, some camera sensors do all the processing inside and only dump the ready-made JPEG (this is not the case of the Zenfone 2). But, in theory all sensors (dumb VGA webcams, point-and-shoot cameras, DSLR professional cameras, smartphone cameras) could be capable shooting in RAW. In smartphones, most of the time, the limitation is on the camera software, not in the hardware. The Zenfone's (and other Intel) raw shooting ability is just a little bit hidden
Click to expand...
Click to collapse
TIL! I've been searching for application that could take RAW format since Camera2 API was released, but all of them are not compatible with Zenfone 2. Looked up to the internet and most of them claimed that the hardware does not support taking photos in RAW format, then I eventually accepted it as a fact.
Thanks for the awesome application.
I try your app on my Zenfone 2 (ZE551ML) with resurrection remix 5.8.3, but it won't install, is your app only for stock rooted firmware??
Sorry,but i can't find any photo i've taken and there is no menu at all
themonyo666 said:
not working it's not compatible on my z00d mokee rom nightly 7.1.2, why?
Click to expand...
Click to collapse
Because the sensor used in the Zenfone 2 ZE500CL is different. Unfortunately, the app needs some specific information about the camera sensor to generate the DNG files. However I'm planning to make the app smarter to get all this specific sensor info automagically so it could work on a wider range of phones
amerli73 said:
I try your app on my Zenfone 2 (ZE551ML) with resurrection remix 5.8.3, but it won't install, is your app only for stock rooted firmware??
Click to expand...
Click to collapse
Hmmm... That's weird. What was the error message during the installation?
dimitrakhs1 said:
Sorry,but i can't find any photo i've taken and there is no menu at all
Click to expand...
Click to collapse
Inside the "DCIM" directory there is another directory called "RawDumper"; the DNG pictures should be there
dreckneck said:
Hmmm... That's weird. What was the error message during the installation?
Click to expand...
Click to collapse
Just this
amerli73 said:
Just this
Click to expand...
Click to collapse
I've got the same result.
Using: Lineage OS 14.1 - Latest Build
dreckneck said:
Inside the "DCIM" directory there is another directory called "RawDumper"; the DNG pictures should be there
Click to expand...
Click to collapse
I know.Still nothing...
This is a bomb! Good quality and a lot of headroom to play with ir RAW developing program (i use raw therapy)
Dots pattern appears in photos taken in RAW mode with OP5 with a third-party apps.
You can have this dots if you taken a RAW photo with Open Camera or a photo in Google Camera mod (who use RAW for HDR+ process). If you take the photo with the stock camera, there is no issue.
It's a bad implementation of the raw stream by Oneplus. They send all signals of the camera when you use the Camera2 API. So, that's why we see this dots (who is the Phase Detection pixels) in the final RAW picture.
I created a bug report in Oneplus forum about this bug. If you can upvote the thread in the link below for more visibility, it could really help for the resolution of this bug.
Thank you.
Link : https://forums.oneplus.net/threads/dots-in-raw-photos-of-third-party-camera-app.690437/
How to shoot both cameras at once and save 40mpx color and 20mpx monochrome pics separately? I just need a way to shoot both cameras at the same time and save both pictures separately.
Any aftermarket app? Any tweak to achieve this goal in the original camera app?
Thx
A Little bump to the question.
Any chance to get shooting both cameras at the same time and saving both archives in raw format?
I doubt it's possible, at least i can't think of a way to do it either with the original camera application or through some third party solution. The original camera application uses the blackwhite sensor along with the 40mpix sensor so it should be impossible. Any third party cameras can usually only access the main 40mpix camera because of a software restriction.
/ Magnus
I have the latest Wichaya's GCam on my Z8. Started having a problem - after taking a picture I click on gallery button (at the bottom right) to view the picture just taken. The picture flashes on the screen for a second then vanishes and the camera preview stays black.
I had this problem before and solved it before in the GCam settings Advanced > Use a third party gallery - turning this off and then back on.
No luck this time. Any Ideas that I could try?
Thanks.