I made a script to backup any partition in our A7:
YOU HAVE TO BE ROOT, TO USE IT.
To run it simply push to the sdcard fire up adb shell to launch the script.
from the computer while connected to the A7 with usb debugging turned on
Code:
adb push PATH_to_the_script/full-backup.sh /sdcard
and
Code:
adb shell
su
cd /sdcard/
sh full-backup.sh
you will see a menu to choose which partition to backup.
As in the attachment.
enjoy, and please hit the thanks button.
the script v1.00
and a terminal screenshot.
Could you use terminal emulator to run this from on the phone thus cutting out the need for a pc?
ultramag69 said:
Could you use terminal emulator to run this from on the phone thus cutting out the need for a pc?
Click to expand...
Click to collapse
yes you could run it directly form the device:
fire up a terminal emulator app:
RUN IT AS ROOT
Code:
su
sh /sdcard/full-backup.sh
thank you. we can backup unmodified firmware via this tool right?
RealSchule said:
thank you. we can backup unmodified firmware via this tool right?
Click to expand...
Click to collapse
So you need to be root to use this script, so not really..
RealSchule said:
thank you. we can backup unmodified firmware via this tool right?
Click to expand...
Click to collapse
The root modifed just the boot partition so you could backup every stock partition but the stock boot... but you could get the stock boot from other places...
manu7irl said:
The root modifed just the boot partition so you could backup every stock partition but the stock boot... but you could get the stock boot from other places...
Click to expand...
Click to collapse
we can backup all images except boot.img with this tool but we can find boot.img from other topic. so we have full stock images right?
If you have root you can backup every single partition. This is not full stock rom! This is for backup and restore easily.
manu7irl said:
If you have root you can backup every single partition. This is not full stock rom! This is for backup and restore easily.
Click to expand...
Click to collapse
So what is the process to restore these images? Is it as simple as flashing in twrp?
Go in TWRP, fire up adb shell on your PC, then do:
Code:
dd if=/sdcard/[backup_folder]/[partition].img of=/dev/block/bootdevice/by-name/[partition_name]
example:
Code:
dd if=/sdcard/backup-part/[date-folder]/system.img of=/dev/block/bootdevice/by-name/system
I will also make a restore part in this little script. If you are intersting in one to do so.
Sure that would be great. Thanks.
manu7irl said:
Go in TWRP, fire up adb shell on your PC, then do:
Code:
dd if=/sdcard/[backup_folder]/[partition].img of=/dev/block/bootdevice/by-name/[partition_name]
example:
Code:
dd if=/sdcard/backup-part/[date-folder]/system.img of=/dev/block/bootdevice/by-name/system
I will also make a restore part in this little script. If you are intersting in one to do so.
Click to expand...
Click to collapse
Been waiting for just that.
as usual, if anything goes wrong, no responsibility etc
The official update tutorial for rooted users doesn't keep app data, only internal storage! This tutorial keeps EVERYTHING
This method allows updating from 4.5.15 (unlocked, encrypted, rooted) to 5.0.2 without any data (sys settings + app data + user data) loss
A FAQ section is present at the end of this post and will be regularly updated.
List of files to download while doing the following steps:
OOS 5.0.2 ROM
Codeworkx TWRP recovery
Latest Magisk
Terms and software used in this guide:
Fastboot / Bootloader = bootloader of the phone, it's a very low level mode of the phone that allows booting into recovery. Can be accessed by using the advanced reboot menu (enable in dev options) or by "adb reboot fastboot".
Recovery = a small operating system on the phone that allows you to do various operations even when the main OS (Android) is broken. This includes flashing ROMs, modifying stuff on the storage, etc. It's the Android swiss army knife. If you can get a phone to boot TWRP, then you can do almost anything.
Magisk = rooting software that uses a systemless method to keep SafetyNet working. Systemless = instead of modifying the system, every change is put in a separate image that is mounted "over" the system. When the system tries to access a file modified by Magisk, instead of reading it from the partition, it reads it from Magisk. It's recommended to use Magisk instead of SuperSU as of 2018.
ADB = tool that allows controlling the phone from your PC through USB. You can use it when you're in Android if USB debugging is enabled in the settings, or when you're in TWRP. Here, we mostly use it for transferring files directly (without MTP) and running commands (using "adb shell")
Note: For this guide you will be required to download and install Magisk. If you don't want your phone to be rooted, then at the end of this guide reboot into TWRP, wipe both caches and re-flash the OS. This will uninstall Magisk and any other root patch. Beware: it will reflash stock recovery, so if you ever want to re-root, you'll need to reboot to fastboot and flash TWRP manually.
Convention for commands that you will have to run:
a command line starting with "C:" means that it should be run on your PC
a command line starting with "~ #" means that it should be run on your phone (through adb shell) while in TWRP
a command line starting with "OnePlus5:/ $" means that it should be run on your phone (through adb shell) while in OxygenOS
Although the commands start with "C:", this is just for readability purposes. You should run everything from inside an empty directory with enough disk space and writing access.
Your phone will have to be plugged in to your PC from the beginning to the end. Also, make sure it has at least 80% battery before beginning, just in case.
I know, the tutorial is huge. This is simply due to the fact that if I just wrote "make a nandroid backup of this and that, flash, and restore the backup while doing this", then some people may encounter problems because not everyone knows how to do a nandroid backup, restore it, etc. Also, there are a lot of things that need to be done precisely that way and not another way, which explains why the tutorial is huge. Also, you may notice that there is a lot of commands to run throughout the tutorial, this is because that way, I'm sure that at the end, you will have done everything like I did it on my phone, so that if you have a problem it's much easier to figure out where it comes from.
Summary of what you need to do (this is only a SUMMARY to give you a preview of what the whole thing looks like, you shouldn't follow it except if you're really an expert since a lot of things need to be done precisely, instead you should follow the easier complete steps below):
Make a Nandroid backup of /data
Backup files on internal storage
Wipe everything (internal storage + /data + system + caches), and then format data (important!)
Push and flash the OOS zip
Wipe caches and reboot (to Oreo!)
When it reboots, make sure everything (features, like Wi-Fi and fingerprint sensor) works. Don't "save anything" though, everything you do will be erased when we'll restore your backup. This is just a "test drive" for Oreo.
Reboot to TWRP, wipe Data and restore the /data backup
Run the three commands to fix Wi-Fi and fingerprints
Reboot (to System) and check everything works (don't do anything, don't change any setting, just make sure it works)
Reboot to TWRP, rename the "Android" folder to "Android_oreo" on sdcard, delete everything else on sdcard and restore your internal files
Rename the freshly restored Android (nougat) folder to "Android_nougat" and rename "Android_oreo" to "Android".
Flash Magisk, wipe dalvik+cache and reboot to System
When in Android, everything should work except some apps won't have their data. This is normal. Open a terminal (either on your phone using Termux or from your PC using adb shell), elevate using su and rename "Android" to "Android_oreo" and "Android_nougat" to "Android" (this is so that it correctly restores permissions)
If everything works fine, delete the "Android_oreo" folder
First, if you have Xposed Framework (systemless or not) installed, uninstall it. Next, if needed, uninstall any Magisk module that is "Nougat-only" to prevent any problems afterwards.
Boot the phone to bootloader/fastboot (either using advanced reboot, or by using volume down button when you start your phone) and boot to the TWRP recovery by doing
Code:
C:\> fastboot boot twrp-3.2.1-0-oreo-8.1-codeworkx-cheeseburger.img
from your PC.
Next, in TWRP, make a backup of /data (using the Backup button). Then, still while in TWRP, run the following commands:
Code:
C:\> adb shell
~ # cd /sdcard
/sdcard # tar cvf twrp.tar TWRP
/sdcard # md5sum twrp.tar
<< md5 checksum of twrp.tar >>
/sdcard # exit
C:\> adb pull -p /sdcard/twrp.tar
When the above command has finished, make sure that the checksum of the received twrp.tar file matches the one previously displayed.
If it doesn't match, delete the file and run adb pull again. Don't continue following this guide until you have received a 1:1 (checksum-wise) backup of /data.
Code:
C:\> adb shell
<< WARNING: dangerous command! double check the following line is correct before pressing enter! >>
~ # rm -rf /sdcard/TWRP
~ # rm /sdcard/twrp.tar
That was for /data. Now, the backup for the internal storage:
Code:
~ # cd /sdcard
/sdcard # du -csh
<< you should see here the total size of sdcard, that'll give you an idea of how long it'll take >>
/sdcard # tar cvf sd.tar element1 element2 element... elementN
<< in the command above, replace "element1..N" by a space-separated list of what you want to have in the backup.
Keep in mind that your list HAS to contain the element "Android" (case is important). It contains the app data.
Let's say for example you want to keep only the photos you have taken (and nothing, nothing else that was on internal storage).
The photos are in the folder DCIM, so the command will look like this:
tar cvf sd.tar Android DCIM
(because you want DCIM, and Android has to be in the list, no matter where)
>>
/sdcard # md5sum sd.tar
<< md5 checksum of sd.tar >>
/sdcard # exit
C:\> adb pull -p /sdcard/sd.tar
When the above command has finished, make sure that the checksum of the received sd.tar file matches the one previously displayed.
If it doesn't match, delete the file and run adb pull again.
Keep in mind that anything you don't put in that list will not be backed up and will be lost!
Now, you have a backup for all the important stuff so we can start doing the real sh*t.
Now, the important step:
Go back to the TWRP home screen, press "Wipe", "Advanced Wipe" and there check "Dalvik / ART Cache", "Cache", "System", "Data" and "Internal Storage". Confirm using the slider at the bottom of the screen. Press the home button, then "Reboot" and "FastBoot". Now, type the same fastboot command as in the previous step to boot the recovery image. You'll enter the recovery as before.
Now, on your PC, in the terminal, type
Code:
adb push -p OnePlus5Oxygen_23_OTA_029_all_1801292040_d71af3d.zip /sideload
(note: here, we are not using "adb sideload", we are really using "adb push"). In TWRP, click Install, in the file manager go to /sideload and select the OOS zip file. Confirm by sliding. If you get an error, go back to home, click Mount and ensure System is not checked. Then try installing again. If it still does not work, reboot to fastboot, type command again, get to the recovery and install again.
When the zip-file is installed, go home, click "Wipe", "Advanced Wipe" and check both caches and confirm. Then, go home, click "Wipe" and then "Format Data". Then, go home, click "Reboot" and then "System". Your phone will now reboot to Oreo. It will take a long time, but do not turn off the phone. Let it run. On my phone, it took on average 2 minutes for that boot.
You'll be greeted by the "first boot" page. It'll ask you if you want to restore a backup or start anew, choose start anew. Connect to your Wi-Fi network and Google account. Follow all the instructions until you get to the home screen. There, make sure everything works (especially Wi-Fi and fingerprint sensor). Don't save your fingerprints yet, they will be erased afterwards. If everything works, you can continue following these instructions. If not, post a comment down there.
Now that you're at the home screen, go in the settings, About Android and click the build number 8 times to enable Developer Options. Go in there and enable advanced reboot. Then, reboot your phone into fastboot/bootloader using the power button. Type the exact same command as before to start TWRP. Once that you are in TWRP, run the following commands:
Code:
C:\> adb push -p twrp.tar /sdcard/
C:\> adb shell
~ # cd /sdcard
/sdcard # tar xvf twrp.tar
/sdcard # cp /data/misc/wifi/WifiConfigStore.xml /sdcard/
In TWRP, click "Wipe", "Advanced Wipe" and check only the "Data" partition. Confirm. Press home, then "Restore" and choose the backup in the list. Confirm to restore. Back to the terminal, we need to run the following commands otherwise Wi-Fi and fingerprints won't work:
Code:
/sdcard # cp WifiConfigStore.xml /data/misc/wifi/
/sdcard # rm /data/misc/wifi/wpa_supplicant.conf
<< WARNING: dangerous command! double check the following line is correct before pressing enter! >>
/sdcard # rm -rf /data/system/users/0/fpdata
/sdcard # rm /data/system/users/0/settings_fingerprint.xml
Note: the command above are ran from your PC in an adb shell while the phone is still in TWRP.
Reboot the phone to system and ensure Wi-Fi and fingerprints are still working. Right now you should already see your old home screen and all your apps, but the internal storage isn't there yet. Reboot in fastboot, run the command to get in TWRP.
Once TWRP has booted, run the following commands:
Code:
C:\> adb push -p sd.tar /sdcard/
C:\> adb shell
~ # cd /sdcard
<< WARNING: dangerous command! double check the following line is correct before pressing enter! >>
/sdcard # rm -rf Alarms Albums DCIM Download Movies Music Notifications Pictures Podcasts Ringtones
/sdcard # ls
<< now, look at the list of files that were printed, and rm anything left that is not called "Android" or "sd.tar"
<< WARNING: dangerous command! double check everything is correct before pressing enter! >>
if when you do rm <the thing> it tells you it's a directory, then do: rm -rf <thething>
if there's a folder called SomeFolder, do "rm -rf SomeFolder"
next, run this:
/sdcard # ls
Android sd.tar <-- expected output
/sdcard # mv Android Android_oreo
/sdcard # tar xvf sd.tar
/sdcard # mv Android Android_nougat
/sdcard # mv Android_oreo Android
If you don't have Magisk somewhere on your sd card, download it and upload it using MTP or adb. Then flash it using the Install button. Clear dalvik/cache and reboot to system.
When the phone has booted (again, it might take time), make sure USB debugging is enabled and run the following commands:
Code:
C:\> adb shell
OnePlus5:/ $ su
<< here, you might see a Magisk screen asking for superuser access. Allow. >>
OnePlus5:/ $ cd /sdcard
OnePlus5:/sdcard $ mv Android Android_oreo && mv Android_nougat Android
Now, try some apps and make sure all the data is there (especially games and Netflix/Hulu/etc). If everything is there, and the phone works properly, go back in the terminal and type:
Code:
OnePlus5:/sdcard $ rm -rf Android_oreo
Optionally, start the TWRP app and flash it, it can always be useful. You can also reboot to fastboot to do that.
Now reboot your phone (normal reboot) one last time.
There, working OOS 5.0.2 / Android 8.0.0 phone with no data loss.
Frequently Asked Questions
How long does the whole thing take?
Highly depends on the amount of data you have on your phone. Since it the USB port only supports USB 2.0, it may take 4 or 5 hours in total.
Will doing this void my warranty?
No.
Will I be able to install future OTA updates using the regular download-reboot-flash-twrp procedure?
Yep. Just use the regular method as you would have on Nougat.
Will I lose my data?
If you follow all the instructions, no. Even if you don't follow them, as soon as you have made a backup of /data and internal storage, then no matter how bad you screw up you could always get a working phone back.
I followed the instructions and now my phone doesn't work
Boot in TWRP, wipe everything, reflash.
questions will be added there in the future
Having WiFi and fingerprint issues
You sure it is a good idea to just delete those files? I would have guessed that I need to replace these (nougat version from backup) with the oreo version to have it working just like before the restore.
Code:
/sdcard # rm /data/misc/wifi/wpa_supplicant.conf
/sdcard # rm -rf /data/system/users/0/fpdata
/sdcard # rm /data/system/users/0/settings_fingerprint.xml
I'm having the issues with wifi and fingerprints. Neither one is working. I'll try to figure out how to fix this.
@zdimension Thanks for this guide, I don't have time to test it yet, but I have a question
pdluke said:
Code:
/sdcard # rm /data/misc/wifi/wpa_supplicant.conf
/sdcard # rm -rf /data/system/users/0/fpdata
/sdcard # rm /data/system/users/0/settings_fingerprint.xml
Click to expand...
Click to collapse
At this point in the procedure, adb shell is still using root (before flashing magisk) ? How is that possible ? Does the adb /sideload preserve root ?
olivier380 said:
@zdimension Thanks for this guide, I don't have time to test it yet, but I have a question
At this point in the procedure, adb shell is still using root (before flashing magisk) ? How is that possible ? Does the adb /sideload preserve root ?
Click to expand...
Click to collapse
These commands should be run while the phone is in TWRP. Also note that adb /sideload is not used here, only adb push.
pdluke said:
You sure it is a good idea to just delete those files? I would have guessed that I need to replace these (nougat version from backup) with the oreo version to have it working just like before the restore.
Code:
/sdcard # rm /data/misc/wifi/wpa_supplicant.conf
/sdcard # rm -rf /data/system/users/0/fpdata
/sdcard # rm /data/system/users/0/settings_fingerprint.xml
I'm having the issues with wifi and fingerprints. Neither one is working. I'll try to figure out how to fix this.
Click to expand...
Click to collapse
If you delete them, they will be generated automatically at the next system boot. But you could also make a backup of those three files before wiping /data, store that somewhere, restore Nougat /data and then restore your backup of those three files. The result would be the same.
Note: actually, not exactly. Erasing the first file won't change anything since it's not used anymore in Oreo, but the two other files contain the fingerprint configuration (list of saved fingerprints). So,
Either you remove the files and you have to save your fingerprints again at next boot
Either you restore them from an Oreo backup and you'll get the fingerprints you had saved during the "first boot" procedure when you rebooted the phone right after flashing the OS
But the result is mostly the same: everything works. Deleting the files ensures you get something clean. If you restore from an Oreo backup I can't guarantee the result (as it may interfere with other files from the Nougat backup).
10 bucks to make a script to do this all for me haha.
@zdimension Thanks for the clarification Another thing you might add to the files to download would be Magisk (optionally). In this kind of guide, I've always find it useful to download everything first.
olivier380 said:
@zdimension Thanks for the clarification Another thing you might add to the files to download would be Magisk (optionally). In this kind of guide, I've always find it useful to download everything first.
Click to expand...
Click to collapse
Oops, forgot to add it
I added the link, and also instructions for how to un-root afterwards for those who would want it.
To improve the guide, here are some ideas :
- You should highlight that rm -rf is a very dangerous command, and that it needs to be checked twice (especially the targeted folder)
- It could be useful to use the du -csh command to check the size of a folder (to estimate the backup time for example).
- As a safety measure, one could md5sum the tar file before and after using adb pull
What do you think ?
olivier380 said:
To improve the guide, here are some ideas :
- You should highlight that rm -rf is a very dangerous command, and that it needs to be checked twice (especially the targeted folder)
- It could be useful to use the du -csh command to check the size of a folder (to estimate the backup time for example).
- As a safety measure, one could md5sum the tar file before and after using adb pull
What do you think ?
Click to expand...
Click to collapse
Thanks for the ideas! I updated the post (and I added a changelog at the bottom for future reference).
Followed guide for successful upgrade from 4.5.15 encrypted, unlocked bootloader w/ Magisk root.
One note, after the first complete wipe and flash of the full ROM, it was getting stuck on first boot and never completed. Discovered that I needed to not just wipe the Data partition but Format it in TWRP, to clear out the old encryption I think. Magisk wouldn't install either until I did this.
@debork thanks for the positive feedback (all the merit goes to @zdimension of course)
@zdimension there are many people in the other thread https://forum.xda-developers.com/oneplus-5/how-to/official-oxygenos-4-5-2-7-1-1-ota-t3627003 that tried (unsucessfully) to upgrade from 4.5.15 to 5.0.1, maybe a link to this topic could be useful for them (if it's not too late).
Regarding the
Go back to the TWRP home screen, press "Wipe", "Advanced Wipe" and there check "Dalvik / ART Cache", "Cache", "System", "Data" and "Internal Storage".
Click to expand...
Click to collapse
I think it should be highlighted in red, since it is the actual "clean flash" (AFAIU, correct me if I'm wrong).
if we only have the BL unlocked non root and stock recovery can we only ota without any loss of data ?
debork said:
Followed guide for successful upgrade from 4.5.15 encrypted, unlocked bootloader w/ Magisk root.
One note, after the first complete wipe and flash of the full ROM, it was getting stuck on first boot and never completed. Discovered that I needed to not just wipe the Data partition but Format it in TWRP, to clear out the old encryption I think. Magisk wouldn't install either until I did this.
Click to expand...
Click to collapse
Thanks for feedback, I will add that to the guide (although it worked with just Wipe for me )
zdimension said:
Thanks for feedback, I will add that to the guide (although it worked with just Wipe for me )
Click to expand...
Click to collapse
Have you rooted the 4.5.15 with Magisk or SuperSU (which is not compatible with Oreo anymore) ?
olivier380 said:
Have you rooted the 4.5.15 with Magisk or SuperSU (which is not compatible with Oreo anymore) ?
Click to expand...
Click to collapse
I stopped using SuperSU when it was sold to that shady company. Also, Magisk is better imo.
quick05 said:
if we only have the BL unlocked non root and stock recovery can we only ota without any loss of data ?
Click to expand...
Click to collapse
Official OnePlus support said that nothing is guaranteed if your bootloader is unlocked. But since you're on stock recovery + non rooted, you could always try. But backup everything first. Some people here on XDA have reported that it doesn't work, though.
Just followed your guide with no problems. Thank you very much!! I can confirm also that you need to format data after the wipe otherwise it gets stuck in a bootloop!
Thanks so much for this. I was able to successfully follow the guide and get upgraded to 5.0.1 without losing any data. In fact, I even messed up one step by failing to include the Android directory in the sdcard.tar backup (perhaps that should be more explicit), but it doesn't seem to have affected everything; all of my apps seem to have retained their data.
A few notes:
1. The file size of twrp.tar was ~14GB but when executing the pull command, it recognized it as only ~1.3 GB. As a result, the pull was not complete until it reached over 1000%. All the more reason to do the md5 check.
2. As others stated, I needed to format the data partition, not just wipe it.
3. I might recommend also including a "summary" version somewhere on what this guide does. Scrolling through the guide the first time, it seemed pretty daunting, but really all that you're doing is: backing up data partition and internal storage; wiping device; flashing Oreo ROM; tweaking a few files; and restoring backed up data and internal storage.
Thank you again so much! Glad to finally be on Oreo.
elight3 said:
Thanks so much for this. I was able to successfully follow the guide and get upgraded to 5.0.1 without losing any data. In fact, I even messed up one step by failing to include the Android directory in the sdcard.tar backup (perhaps that should be more explicit), but it doesn't seem to have affected everything; all of my apps seem to have retained their data.
A few notes:
1. The file size of twrp.tar was ~14GB but when executing the pull command, it recognized it as only ~1.3 GB. As a result, the pull was not complete until it reached over 1000%. All the more reason to do the md5 check.
2. As others stated, I needed to format the data partition, not just wipe it.
3. I might recommend also including a "summary" version somewhere on what this guide does. Scrolling through the guide the first time, it seemed pretty daunting, but really all that you're doing is: backing up data partition and internal storage; wiping device; flashing Oreo ROM; tweaking a few files; and restoring backed up data and internal storage.
Thank you again so much! Glad to finally be on Oreo.
Click to expand...
Click to collapse
Thanks for the feedback! I'll add a summary to the guide.