Database Develop

[TUTORIAL] Build CyanogenMod 11 on a VPS or Dedicated Server

Introduction
CyanogenMod 11 is a custom firmware used on android phones and tablets. But, you already know that, chances are you've used it. Building ROMs may seem scary and confusing at first because of the code and terminals, but in reality, it's not that difficult. In this thread, we're going to be discussing how to build CyanogenMod on a VPS or dedicated server. I say a server and not a desktop machine because this is going to be geared towards keeping things simple. When you have only one shell to work with, it's usually easier.
Vocabulary
A couple words you may be unfamiliar with will be described here.
- shell: where you type in the commands on your VPS/dedicated server
- VPS: Virtual Private Server, a type of server that is virtualized from physical hardware
- Dedicated Server/dedi: a dedi is a physical server, it can be an old computer, or (recommended) a server from a server host/data center like Incero or Colocrossing
- Linux: the operating system in which we will build android
- SSH: Secure SHell, the world-standard protocol used to control Linux servers
- bash: the shell we're going to use
Linux Commands
In order to control our server, we won't be using a GUI (Graphical User Interface) where you can click things to make things happen, we will be using SSH. SSH sessions are just a prompt where you type in commands. This is where people get scared but calm the f*ck down. It's not as scary as movies make it seem. Here's a rundown of some basic commands we'll be using.
- cd: change directory, change the folder we are working in, equivalent of changing folders in windows explorer
- mkdir: make directory, makes folders
- cp: copy, it copies files
- mv: move, it moves files
- rm: remove, it deletes files
- ls: list files, it's like dir for windows
- curl: downloads files from URLs
- adduser: adds users, linux has users just like windows
- apt-get: package manager, it's used to install things. In Windows, you usually install things from .exe or .msi files. In Linux, you generally use a package manager. This will be explained more later.
- nano: our text editor of choice for this tutorial. I chose nano because it's very simple and easy to use for a beginner.
Basic Intro to Linux
A few things that I'm going to discuss about Linux are directory structure as well as file structure. In Windows, we have our C: drive, which is where pretty much everything stays. In Linux, we have /, it's the same thing, except it's represented as a slash. If I wanted to edit C:\cookies\morecookies.txt in Linux, the same file path would be /cookies/morecookies.txt. A couple other things about Linux are that the file system is case sensitive, meaning I can have two files named Android and android in the same directory and they won't conflict, whereas in Windows it would. Linux also uses / (forward-slash) instead of \ (backslash) in the directory structures.
Now, our shell is interactive, meaning it's not just a box we type things into, we can use it in different ways. One of the things you should know about bash is ~ (the tilda). ~ in Linux refers to our home directory. Every user has their own home directory (by default at least). Standard users' homes are stored in /home. If my username is tanmay, my home directory would be /home/tanmay/. If I'm logged in as tanmay, ~/android refers to a file at /home/tanmay/android. If I'm logged in as bob, ~/android refers to /home/bob/android. If you don't understand all this completely, I don't expect you to, don't worry. This was just to give you a brief introduction to Linux.
Buying a VPS
I'm guessing most of you already know how to do this, so I'll put it in a spoiler for those who don't need it loading loads of images.
For this tutorial, we're going to use a DigitalOcean VPS, you can sign up here: https://www.digitalocean.com/?refcode=2050223a4edc
After you sign up for DigitalOcean, head to the billing tab and enter in credit card info or put some money in through PayPal to get yourself some credits. Once done, click the big green Create button. The hostname can be anything you want, I usually just make it "android" or "build". Now, you have a choice here, you can spend less and wait longer for builds, or spend more and wait less. I recommend one of these:
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Any lower and the build will likely just fail, and any higher is just unnecessary. 16GB is plenty, and 8GB will work just fine. You can go with 4GB if you really can't afford it, but I don't recommend it.
In Select Image, choose Ubuntu 12.04.4 x64 and click Create Droplet. Wait for it to spin up, and check your email. You'll have an email like this:
We'll use these details later.
Getting into our server with SSH
In order to SSH into our server, we need a client. The most common client for Windows is PuTTY. PuTTY is free and open source, it can be downloaded here: http://the.earth.li/~sgtatham/putty/latest/x86/putty.exe (direct link). Open it and you should see this:
Under IP, type in your server's IP. If you used DigitalOcean, it should be in your email. Click Open and when it says login as:, enter "root" without quotes, and then when it asks for password, use the one supplied in the email.
If you're on Mac or Linux, you can open a terminal and type in:
Code:
ssh [email protected] # replace server.ip with the IP of your server, if you used DigitalOcean, it's in the email
Use the password supplied in the email.
Installing The Necessary Packages
Once we're in, we need to update the server's current packages to the latest ones, we can do so with the following commands.
Code:
apt-get update # update the package sources list
apt-get -y upgrade # upgrade all the current packages
Now that that's done, we can install the libraries and packages we need.
Code:
apt-get -y install git gnupg flex bison gperf \
zip curl libc6-dev libncurses5-dev:i386 x11proto-core-dev \
libx11-dev:i386 libreadline6-dev:i386 libgl1-mesa-glx:i386 \
libgl1-mesa-dev g++-multilib mingw32 tofrodos \
python-markdown libxml2-utils xsltproc zlib1g-dev:i386
apt-get -y install build-essential schedtool screen python-software-properties
Android relies on many libraries and compilers, so we can't build it without them.
Oracle Java is closed source, so we have to do some other things to install it.
Code:
add-apt-repository -y ppa:webupd8team/java # add third party source to the package manager's list
apt-get update # refreshes the sources list to include the one we just added
apt-get -y install oracle-java6-installer
While installing oracle-java6-installer, you will be prompted to accept the Oracle Sun Java license agreement whatever, use your arrow keys to select accept then click enter.
Adding Swap
Even with 8GB of RAM, you'll probably run out of memory, so we use swap. Swap is hard drive space that acts as if it's memory for things that aren't important to the host at the current time. We'll need this, because android takes a LOT of RAM to compile.
Code:
fallocate -l 8G /swapfile # create an empty file that's 8GB in size called /swapfile
mkswap /swapfile # format the file to be swap
swapon /swapfile # turn the swapfile on
Adding a User
Now that we have all the required packages to build, let's add a user to build under.
Code:
adduser tanmay # replace tanmay with your username of choice, must be lower case
Enter the password when it prompts for one, then do
Code:
login tanmay # replace tanmay with the username above and enter the password
You should see something like this:
Note: Disregard the system restart message, it's not necessary for our purposes.
You may have noticed that the prompt changed from "[email protected]:~#" to "[email protected]:~$". This shows we're in our new account.
Setting up repo and Downloading the CyanogenMod 11 Source
repo is the tool supplied by Google used to manage the android source code. Since it's used by AOSP, pretty much every ROM uses repo to manage their source. To install repo, we need to make a folder called bin (binary), where we'll store it. This is mainly for organization purposes. After that, we'll download repo to that folder.
Code:
mkdir ~/bin # create a folder called bin in our home directory
curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo # download repo and store it in ~/bin/repo
chmod +x ~/bin/repo # mark the file as executable, so we can run it
export PATH=$PATH:~/bin # explained below
The last line may seem scary, but all it does is take the current PATH variable and add ~/bin to it. If you do
Code:
echo $PATH
you get
Code:
[email protected]:~$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/home/tanmay/bin
You can see /home/tanmay/bin at the end of it. This allows us to use the repo command anywhere in the filesystem, not only in the ~/bin directory.
Now let's make a folder for our source code.
Code:
mkdir cm-11.0 # make a folder called cm-11.0
cd cm-11 # move into the directory we just made
You should see your shell prompt change like so:
Code:
[email protected]:~/cm-11.0$
This shows us what directory we are in for reference. Now, let's initialize repo in this folder with the CyanogenMod 11 repo.
Code:
git config --global user.name "Your Name" # Necessary for repo to init
git config --global user.email [email protected] # same as above
repo init -u https://github.com/CyanogenMod/android.git -b cm-11.0
When you get prompted if you want color or not, type "y" then press enter.
Now we need to actually download the source.
Code:
repo sync
This takes... a long time. Sit back and relax. If you're using a server, it'll probably take around 10-15 minutes, if you're on a home line, expect a few hours as the source is around 8GB large.
Getting Proprietary Blobs
Since every phone has different hardware, each one requires different drivers. The Nexus 4 (or mako) has it's own, along with every other phone. We need to download these so that CyanogenMod can compile properly. We can incorporate these into our local build using the local_manifests folder.
Code:
cd ~/cm-11.0/.repo # cd into our repo folder
mkdir local_manifests # make a folder called local_manifests
nano local_manifests/roomservice.xml
On the last line, you'll see an editor (nano) open up, this is editing ~/cm-11.0/.repo/local_manifests/roomservice.xml. The name roomservice.xml is just a formality, it can be anything you want. Anyways, inside the file, paste in the following:
Code:
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<manifest>
<project name="TheMuppets/proprietary_vendor_lge" remote="github" revision="cm-11.0" />
</manifest>
Once you enter that in, press CTRL + X, Y, then enter. This will save and exit the file.
Next run
Code:
cd ~/cm-11.0/ # bring us back to the parent directory of the source
repo sync # sync again to download the new project we added
Don't worry, this sync won't take hours like the last one, this shouldn't take more than a minute or two.
Getting Prebuilts and Building!
We're getting to the actual build! What we have to do now is get the prebuilt apps that CyanogenMod comes with. We can do this with the following command.
Code:
~/cm-11.0/vendor/cm/get-prebuilts
Once those download, run the following.
Code:
export USE_CCACHE=1 # ccache is compiler cache, it makes future builds MUCH faster, this is not necessary if you're only going to build once
source build/envsetup.sh # load the commands supplied by CyanogenMod used to build
lunch cm_mako-userdebug # generate the Makefile, the instructions for the compiler on how to build the ROM
ARE YOU READY?! THIS IS IT.
Code:
mka bacon # yes, bacon
Aaaaaaaaaaaaand wait. Wait a lot. This is going to take a LONG time.
Downloading the ROM
To download your ROM, download WinSCP from here: http://winscp.net/eng/download.php
Once installed (or just downloaded), open it and enter in the same credentials that you used in PuTTY except use the username and password we added ("tanmay" in my case) instead of root. Click login, and click Okay for messages that you get prompted for. Once you're in, open the cm-11.0 folder, then out/product/target/mako/. Inside that folder, you should see a zip called cm-mako-something.zip. Drag this file onto your desktop or a folder of your choice and then flash it.
CONGRATULATIONS YOU BUILT CYANOGENMOD 11 FOR THE NEXUS 4
Wasn't so bad, was it?
I hope this guide has helped you, if it has, please press the Thanks button below.
Thanks for reading!

awesome guide bt can u help me regarding mtk ........how can i add device tree and vendor manually

Nice tutorial. Just want to add that if your going with DigitalOcean you have the choice of deploying an instance that has Docker preinstalled on Ubuntu. In that case following this tutorial is much simpler and easier.
http://forum.xda-developers.com/showthread.php?t=2650345
Thanks !

[email protected] said:
awesome guide bt can u help me regarding mtk ........how can i add device tree and vendor manually
Click to expand...
Click to collapse
Hi bro @[email protected]! Check it may be useful:
https://github.com/axet/android_device_mt6592
https://github.com/axet/android_vendor_mt6592
Regards.
hyperion70.

Reduce shell code (and/or debug)

Could you give me a little hand to reduce this code and make it more bearable because they (codes) will be a series of consecutive actions, and in a single command line in Tasker.
What I want to do in short?
1. Compress a file (or folder)
2. Add a name to the final file
3. Apply compression to the generated file
4. Delete ONLY contents of the folder that ended compress
5. Move the file was created before to the empty folder
Here is the code:
Code:
cd /xxx && tar -cf xxx_%DATE.xxx "$$$" && gzip -x xxx_%DATE.xxx && cd "###" && rm -r * .* && cd /xxx && mv -f xxx_%DATE.rar.gz "€€€"
And this is the description that I will offer users who use it:
Code:
[cd /'xxx'] = path to backup folder (parent dir) ; ['xxx'_%DATE] = file name (no name spaces) ; ['$$$'] = folder with files to back up ; [gzip -'x'] = compression level (1-9) ; [%DATE.'xxx'] = files container (.zip/.rar/.tar) ; ['###'] = same as -> '$$$' ; ['€€€'] = same as -> '###' & '$$$'

Mmmm, I'm sure you have several questions, so I will try to answer all possible question you could ask:
- What is the path that you think to work on?
-> If possible, in the external memory (SD Card)
- Why did you use in all commands the '&&' concatenation?
-> Just because I don't want to run the following command if the above fails for X reason. Just for security.
- What the variable '% DATE' does in your generated files?
-> All the code you see above is part of a larger task that will be used to create backups of various things, in short: BACKING UP. Well I include that variable because I want Tasker to add to the file name the date when the backup was created.
- Why do you offer the ability of choose the container file extension?
-> At first I thought it would not be possible and that failure or something would occur, but as I tested it, nothing happened. So I offer users the ability to generate the container they want. And as the final file (.gz) after compression, you can easily open it, so...
- Why do you use the quotes in some parts of the code that refer to a directory to compress, or to dump the created contents (move)?
-> Not really would take, well at least that I've experienced. But I decided to added it because, in my case, I have folders which their names have spaces, and without adding those quotation marks ("...") I can not find the directory that want to access, I guess it can happen to other people.
- When you begun using the command to create file container, like moving file, why do you position yourself in the directory first and then do the action, rather than perform all in the same code?
-> This is what I've tried so far, but do not know why I couldn't. Every combination I tested: FAILED, then I decided to separate positioning directory commands of leading compression actions, moving files, etc.
Well, I think the rest of you get the idea of that or how it works, but if not, ask to me.

Problems write to logfile.

At the first run python script nothing is written to the log file "cpu_temp.csv". When I run the second time then the data is written to the log file. The script is to extract the temperature from the temperature sensor MCP9808. I know the data is first written to "w1_slave" under the folder "/sys/bus/w1/devices/28-0115a4f575ff". The data for the log file are read always from this file. Can someone help me forwarding such that at the first startup script the data read from the sensor is written to the log file? I want the script started via cron. But since the first run doesn't work nothing has been logged.
Python script for reading data from MCP98808 and write to log file "cpu_temp.csv"
import subprocess
import logging
import time
from time import sleep, gmtime, strftime
import smbus
#Constant things, that don't change during run.
t_reg = 0x05
address = 0x18
bus = smbus.SMBus(1) # change to 0 for older RPi revision
def get_temp():
#The reading variable changes every time you run get_temp()
reading = bus.read_i2c_block_data(address, t_reg)
t = (reading[0] << 8) + reading[1]
temp = t & 0x0FFF
temp /= 16.0
if (t & 0x1000):
temp -= 256
return(temp)
with open("cpu_temp.csv", "a") as log:
while True:
temp = get_temp()
log.write("{0},{1}\n".format(strftime("%Y-%m-%d %H:%M:%S"),str(temp)))
time.sleep(10)
Output file cpy_temp.csv
2017-02-04 15:15:38,22.1875
2017-02-04 15:15:48,22.125
2017-02-04 15:15:58,22.125
2017-02-04 15:16:08,22.125
2017-02-04 15:16:18,22.125

Lowieke19 said:
At the first run python script nothing is written to the log file "cpu_temp.csv". When I run the second time then the data is written to the log file. The script is to extract the temperature from the temperature sensor MCP9808. I know the data is first written to "w1_slave" under the folder "/sys/bus/w1/devices/28-0115a4f575ff". The data for the log file are read always from this file. Can someone help me forwarding such that at the first startup script the data read from the sensor is written to the log file? I want the script started via cron. But since the first run doesn't work nothing has been logged.
Python script for reading data from MCP98808 and write to log file "cpu_temp.csv"
import subprocess
import logging
import time
from time import sleep, gmtime, strftime
import smbus
#Constant things, that don't change during run.
t_reg = 0x05
address = 0x18
bus = smbus.SMBus(1) # change to 0 for older RPi revision
def get_temp():
#The reading variable changes every time you run get_temp()
reading = bus.read_i2c_block_data(address, t_reg)
t = (reading[0] << 8) + reading[1]
temp = t & 0x0FFF
temp /= 16.0
if (t & 0x1000):
temp -= 256
return(temp)
with open("cpu_temp.csv", "a") as log:
while True:
temp = get_temp()
log.write("{0},{1}\n".format(strftime("%Y-%m-%d %H:%M:%S"),str(temp)))
time.sleep(10)
Output file cpy_temp.csv
2017-02-04 15:15:38,22.1875
2017-02-04 15:15:48,22.125
2017-02-04 15:15:58,22.125
2017-02-04 15:16:08,22.125
2017-02-04 15:16:18,22.125
Click to expand...
Click to collapse
When I run it line by line in the command line, this part here is having the issue.
log.write("{0},{1}\n".format(strftime("%Y-%m-%d %H:%M:%S"),str(temp)))
I hope it accented the coma before the str(temp))).
That is what my interpreter is pointing at where the issue may be. Some times it is easier to make output = "{0},{1}\n".format(strftime("%Y-%m-%d %H:%M:%S") + "," + str(temp)
then log.write(output)
Give that a shot for your debugging. I didn't fully run the code, I just gave the variables some fake values and tried it out!
Good Luck.

Writing problems to log file Python Raspberry pi 2

By the first run python script nothing is written to the log file "cpu_temp.csv". When I run the second time then the data is written to the log file. The script is to extract the temperature from the temperature sensor MCP9808. I know the data is first written to "w1_slave" under the folder "/sys/bus/w1/devices/28-0115a4f575ff". The data for the log file are read always from this file. Can someone help me forwarding such that at the first startup script the data read from the sensor is written to the log file? I want the script started via cron. But since the first run doesn't work nothing has been logged.
Python script for reading data from MCP98808 and write to log file "cpu_temp.csv"
import subprocess
import logging
import time
from time import sleep, gmtime, strftime
import smbus
#Constant things, that don't change during run.
t_reg = 0x05
address = 0x18
bus = smbus.SMBus(1) # change to 0 for older RPi revision
def get_temp():
#The reading variable changes every time you run get_temp()
reading = bus.read_i2c_block_data(address, t_reg)
t = (reading[0] << 8) + reading[1]
temp = t & 0x0FFF
temp /= 16.0
if (t & 0x1000):
temp -= 256
return(temp)
with open("cpu_temp.csv", "a") as log:
while True:
temp = get_temp()
log.write("{0},{1}\n".format(strftime("%Y-%m-%d %H:%M:%S"),str(temp)))
time.sleep(10)
Output file cpy_temp.csv
2017-02-04 15:15:38,22.1875
2017-02-04 15:15:48,22.125
2017-02-04 15:15:58,22.125
2017-02-04 15:16:08,22.125
2017-02-04 15:16:18,22.125

Android Studio 4.1.1 has built- in compilation failures

I have installed the latest Android Studio onto a new machine, and have installed Flutter plugin into the new installation. I then created a Flutter project, taking all the default settings.
After creating the project, I set up an emulator and tried to compile and run the flutter app. Unfortunately I get the following error:
Code:
FAILURE: Build failed with an exception.
* Where:
Build file 'C:\Users\rbrown3\AndroidStudioProjects\flutter_app\android\build.gradle' line: 26
* What went wrong:
A problem occurred evaluating root project 'android'.
> A problem occurred configuring project ':app'.
> Failed to notify project evaluation listener.
> Uninitialized object exists on backward branch 142
Exception Details:
Location:
com/android/build/gradle/internal/pipeline/VariantInfoImpl.<init>(Lcom/android/build/gradle/internal/scope/VariantScope;)V @200: goto
Reason:
Error exists in the bytecode
Bytecode:
...bytecode listing omitted for brevity...
Note that I also took a look at the build.gradle file that contained the offending line . Line 26 just contains:
Code:
project.evaluationDependsOn(':app')
I did not edit that line at all. In fact, I did not change anything, not even in the main.dart file.
Have I found a bug in the Flutter plugin? Or is it an Android Studio bug? Why does a pristine installation of Android Studio fail to compile default code due to a flaw in a gradle build file??? More importantly, what is the flaw causig these compilation failures and how do I fix it?