ota_update

Foreword:

Three upgrade methods

First, classify Android upgrades into categories, namely
Non-A/B system updates
A/B (seamless) system updates
Virtual A/B upgrade (Virtual A/B overview)

Non-A/B System Updates

The official website documents are as follows:
https://source.android.com/docs/core/ota/nonab?hl=zh-cn
Non-A/B upgrade is the traditional Android system update method. The system updates the system image file during operation, and after the update is completed, the device needs to be restarted to apply the update.
Workflow:

1. Download the update package: After receiving the update notification, the user downloads the OTA (Over-The-Air) update package.
2. Prepare to update: After the download is completed, the system will decompress the update package and prepare it for installation.
3. Install updates: The device enters recovery mode, the system stops normal operation, and the update package is installed.
4. Restart the device: Once the update is complete, the device reboots and the update is applied.

advantage:

• Implementation is relatively simple as no additional partitioning or complex management logic is required.

shortcoming:

• The device is unusable during the installation process and the update process may take a long time.
• If a failure occurs during the update process, it may prevent the device from booting properly and must be repaired through manual recovery mode.

A/B (Seamless) System Updates

The official documentation is as follows:
https://source.android.com/docs/core/ota/ab?hl=zh-cn
Due to the limitations of non-A/B upgrades, Android has introduced a new OTA upgrade method A/B upgrade (also called seamless update) starting from 7.0.

Overview:

• A/B system upgrade achieves seamless updates by creating two system partitions (A and B).
• While one partition is in use, another partition can be used for updates, thereby reducing device unavailability during the update process.

Workflow:

1. Download the update package: After receiving the update notification, the user downloads the OTA update package.
2. Update the spare partition: The system installs the update package to an unused partition in the background (for example, the currently used partition is A, and the update package is installed to partition B).
3. Switch partition: Once the update is complete, the system will configure the bootloader to boot from the updated partition (for example, switch from A to B).
4. Restart the device: The device restarts and boots from the updated partition.

advantage:

• Seamless updates: Updates occur in the background and users can continue to use the device, reducing inconvenience during the update process.
• More secure: If a problem occurs during the update process, you can roll back to the previous partition to ensure device availability.

shortcoming:

• More storage space is required as two system partitions are required.

Virtual A/B Upgrade (Virtual A/B Overview)

The official documentation is as follows:
https://source.android.com/docs/core/ota/virtual_ab?hl=zh-cn
Since A/B upgrade also has disadvantages, virtual A/B upgrade was introduced.

Overview:

• Virtual A/B upgrade combines the advantages of A/B and non-A/B upgrade, reducing storage space requirements through logical partitioning and dynamic partitioning.
• Introduced in Android 10 and above, seamless updates are achieved by using dynamic partitions without the need for actual physical A/B partitions.

Workflow:

1. Download the update package: After receiving the update notification, the user downloads the OTA update package.
2. Create and update dynamic partitions: The system creates or updates dynamic partitions (using logical partitions) in the background, which are supported by the kernel and mapped in memory.
3. Switch partition mapping: After the update is completed, the system will update the partition mapping in memory to point to the updated logical partition.
4. Restart the device: The device restarts and loads the updated partition.

advantage:

• Seamless updates: Users can update in the background without stopping device operations.
• More efficient storage utilization: Using dynamic partitioning reduces the storage space required by traditional A/B partitioning.

shortcoming:

• The implementation complexity is higher and requires kernel and system support.

The detailed processes of these three can be found below.

[It is enough to read this article for Android OTA upgrade-CSDN Blog (2024_6_17 09_40_23).html](https://blog.csdn.net/m0_56255097/article/details/137125100?spm= 1001.2101.3001.6650.4&utm_medium=distribute.pc_relevant.none-task-blog-2~default~baidujs_baidulandingword~Ctr-4-137125100-blog-135250601.235%5Ev43%5Epc_blog_bottom_relevance_base4&depth_1-utm_ source=distribute.pc_relevant.none-task-blog-2~ default~baidujs_baidulandingword~Ctr-4-137125100-blog-135250601.235%5Ev43%5Epc_blog_bottom_relevance_base4&utm_relevant_index=9)

Determine which way to upgrade your Android system

1. Check the partition structure
   Use adb shell command

adb shell ls /dev/block/by-name/

You will see a list of partitions on the device. Devices for A/B system upgrade usually have a pair of partitions (such as system_a and system_b), while devices for non-A/B upgrade only have one system partition (such as system). like:

console:/ # ls /dev/block/by-name/
boot_a       cri_data  logo          odm_ext_a  rsv       vbmeta_system_a
boot_b       dtbo_a    metadata      odm_ext_b  super     vbmeta_system_b
bootloader   dtbo_b    misc          oem_a      tee       vendor_boot_a
bootloader0  env       mmcblk0       oem_b      userdata  vendor_boot_b
bootloader1  factory   mmcblk0boot0  param      vbmeta_a
cache        frp       mmcblk0boot1  reserved   vbmeta_b

This means it is an A/B system upgrade

2. Check system properties
   Use adb shell command

adb shell getprop ro.build.ab_update

If the output is true, it means that the device uses the A/B system upgrade.
If the output is empty or false, it means the device uses non-A/B upgrade.

Determine which partition is currently used for A/B upgrade

Use adb shell command

cat /proc/cmdline

This is a command used to display Linux kernel boot parameters. On Android devices, it can be used to examine the command line arguments passed to the kernel when the device is booted. Boot parameters may contain information about device characteristics and supported features, such as whether dynamic partitioning is enabled.
androidboot.slot_suffix=_a indicates that the a partition is currently used
androidboot.dynamic_partitions=true indicates that the current device supports dynamic partitioning, indicating that the device may use virtual A/B upgrade.

OTA upgrade package

Production of OTA upgrade package

The OTA upgrade package will be automatically built when we build the Android system, as long as we enter the make otapackage command. He actually used the ota_from_target_files tool provided in build/make/tools/releasetools. We can also use this tool manually to make it. The original zip file (target-files.zip) when the system was built is required during production.
The location of the original zip file is:
out/target/product/[project name]/obj/PACKAGING/target_files_intermediates/ directory

. build/envsetup.sh && lunch tardis-eng
mkdir dist_output
make dist DIST_DIR=dist_output


make  otapackage

Contents of OTA upgrade package

Unzip the OTA upgrade package and the results are as follows:

$ tree -l full-ota
 full-ota
├── care_map.pb
├── META-INF
│   └── com
│       └── android
│           ├── metadata
│           └── otacert
├── payload.bin
└── payload_properties.txt

3 directories, 5 files

1. payload.bin is the data file to be updated by the system. payload_properties.txt contains some attribute information of the upgrade content, as follows:

$ cat full-ota/payload_properties.txt
FILE_HASH=XuFSKnR3J7i9e5g3rd8kc3WM4/hbjnRDmEQDloWtN34=
FILE_SIZE=358295010
METADATA_HASH=uRar88FujcLyXEOF/JLOx2EE2rwFE7kuM1GLYFuYfnA=
METADATA_SIZE=29432

The information in payload_properties.txt will be used during the upgrade.

2. META-INF/com/android/metadata is used to describe the metadata file of the software package

ota-property-files=payload_metadata.bin:1490:36805,payload.bin:1490:67848,payload_properties.txt:69396:150,care_map.pb:809:634,metadata:69:693   
ota-required-cache=0
ota-streaming-property-files=payload.bin:1490:67848,payload_properties.txt:69396:150,care_map.pb:809:634,metadata:69:693
ota-type=AB
post-build=RB56/ohm/ohm:11/RD2A.211001.002/eng.xxx.20240612.134531:userdebug/release-keys
post-build-incremental=eng.xxxxxx.20240612.134531
post-sdk-level=30
post-security-patch-level=2021-10-01
post-timestamp=1718171125
pre-build=xxxxx/ohm/ohm:11/RD2A.211001.002/eng.xxxx.20240612.134531:userdebug/release-keys
pre-build-incremental=eng.xxxxx.20240612.134531
pre-device=ohm

The pre-device, pre-build-incremental and pre-build values ​​define what state the device must be in before an OTA package can be installed.
The post-build-incremental and post-build values ​​define the expected state of the device after the OTA package is installed.
The values ​​of the pre- and post- fields are derived from the following corresponding build properties.

• The pre-device value is derived from the ro.product.device build property.
• The pre-build-incremental and post-build-incremental values ​​are derived from the ro.build.version.incremental build property.
• The pre-build and post-build values ​​are derived from the ro.build.fingerprint build property.

3. MANIFEST.MF CERT.SF CERT.RSA means that the OTA upgrade package has been signed. There are these files on the online blog, but these files are not in the OTA package I compiled, but the OTA package is indeed signed. It's signed.

Upgrade using update_engine_client

The A/B system will include the upgrade application update_engine_client in debug mode
Put the payload.bin file somewhere that can be accessed through http, and then call update_engine_client on the command line to upgrade

 bcm7252ssffdr4:/ # update_engine_client \
 --payload=http://stbszx-bld-5/public/android/full-ota/payload.bin \
 --update \
 --headers="
   FILE_HASH=ozGgyQEcnkI5ZaX+Wbjo5I/PCR7PEZka9fGd0nWa+oY= 
   FILE_SIZE=282164983
   METADATA_HASH=GLIKfE6KRwylWMHsNadG/Q8iy5f7ENWTatvMdBlpoPg= 
   METADATA_SIZE=21023 
 "

The headers option needs to be filled in with the contents of the payload_properties.txt file.
Except for the production of the upgrade package that is different between the incremental package upgrade and the complete package upgrade, the content of the generated upgrade package file is the same, and the upgrade operation using update_engine_client is also exactly the same.

Incremental package

The incremental upgrade package is generated by comparing the differences between the existing basic package and the original basic package. That is, the OTA package has a specific application background (used between two incremental packages), and upgrading with the incremental package will not format it. system partition only rewrites the contents of some storage segments. During the upgrade process, the upgrade script (opening the upgrade package) will detect the fingerprint to ensure that the upgrade package is applied correctly. The fingerprint attribute exists in build.prop. You can enter the root path through adb shell and view this attribute (or getprop) through cat build.prop. The content is in the script META-INF/com/google/android/updater-script of the incremental package.
Production of incremental packages
In the Android source code root directory:

ota_from_target_files i old.zip new.zip update.zip

or

./build/tools/releasetools/ota_from_target_files -i old.zip new.zip update.zip

Notice:

1. The location of the original OTA upgrade package is in the out/target/product/[project name]/obj/PACKAGING/target_files_intermediates/ directory
   It is an intermediate product after using the command make otapackage and is the most original upgrade package.
2. The first method is officially recommended, and the second method is widely seen in blogs. However, in actual operation, I found that the second method will report an error:

Error: Unable to access jarfile build/make/tools/framework/signapk.jar

I checked and there is no such directory. There should be a problem with the definition of certain paths in the file. It is recommended to use the second one.
Here I found a solution online:
https://blog.csdn.net/Tiger99111/article/details/126522501
Add the make signapk command before make otapackage. Its function is to compile the signapk into the incremental environment. Then the compilation will be successful. The steps are the same as above.

Use of incremental packages

Except for the production of the upgrade package that is different between the incremental package upgrade and the complete package upgrade, the content of the generated upgrade package file is the same, and the upgrade operation using update_engine_client is also exactly the same. Therefore, the usage will not be introduced here.

Contents of incremental package

Unzip the update incremental package, the results are as follows:

 tree -l update-ota/
update-ota/
├── care_map.pb
├── META-INF
│   └── com
│       └── android
│           ├── metadata
│           └── otacert
├── payload.bin
└── payload_properties.txt

3 directories, 5 files

Note

1. Using incremental package upgrade may not update the system build time (ro.build.date)
   The reason is explained in this commit below.
   https://android.googlesource.com/platform/external/libchrome/+/8b7977eccc94f6b3a3896cd13b4aeacbfa1e0f84

Instead, pull the build date of the system from the ro.build.date
system property. Then this library will be identical as long as the
sources and dependencies don’t change, and we won’t have to update it
on every OTA.

As long as the sources and dependencies don't change, the library will be the same and therefore not updated.

Partition

On Android devices, system storage is usually divided into multiple partitions, with each partition storing different types of data and code. These partitions can vary on different devices, but generally include the following main partitions:
Boot partition (boot):

• Contains the kernel and initial boot environment.
• Used to boot the device.

System partition (system):

• Contains the core parts of the operating system (such as system applications and libraries).
• This is the main component of the Android system.

Vendor Partition (vendor):

• Contains device-specific binaries and HAL (Hardware Abstraction Layer).
• Make the system and hardware work together.

Userdata partition (userdata):

• Stores user data and application data.
• This partition will be wiped when restoring to factory settings.

Cache partition (cache):

• Store temporary files and OTA update packages.
• Typically used for caching during system updates.

Recovery Partition (recovery):

• Contains recovery mode images for performing recovery operations such as factory reset and OTA updates.

A/B partitions and slots

In order to achieve seamless updates, Android introduces an A/B system update mechanism. This mechanism maintains two system partitions on the device: one for the system currently in use and one for receiving updates. The data partition is now used to store downloaded OTA update packages, while the recovery image code is located on the boot partition. All partitions used for A/B updates should be named as follows (slot names are always a, b, etc.): boot_a, boot_b, system_a, system_b, vendor_a, vendor_b . The main features of the A/B system update are as follows:

1. Partition slot:

• Each partition has two slots, usually called slot A and slot B.
• For example, the system partition includes system_a and system_b, and the boot partition includes boot_a and boot_b.

2. Seamless updates:

• OTA updates will be installed to the alternate slot instead of the currently used slot.
• Once the update is complete, the device will reboot and switch to the updated slot.
• If the update fails, the device rolls back to its original slot, ensuring system availability.

3. Update process:

• Download the OTA update package.
• Write the update package contents to the partition of the spare slot.
• Once the update is complete, switch to the alternate slot and restart the device.

Virtual A/B System Update

Virtual A/B system update is a technology that combines the A/B update mechanism with the traditional update mechanism, aiming to reduce disk space usage while still providing a seamless update experience. Key features of the virtual A/B system update include:

1. Dynamic partitioning:

• Uses dynamic partitioning technology, allowing partitions to be resized during the update process.
• Dynamic partition management is achieved through Logical Volume Manager (LVM).

2. Reduce space usage:

• Compared with traditional A/B system updates, virtual A/B system updates reduce the space required for duplicate storage partitions.
• Dynamic partitions are created or resized only when needed, optimizing storage utilization.

3. Update process:

• Similar to A/B system updates, but uses dynamic partitioning to reduce space requirements.
• The OTA update package is applied to the dynamic partition and switches to the new partition when completed.

Source code demonstration: SystemUpdater

The source code of system A/B upgrade is closely related to UpdateEngine. For the official demo code of A/B upgrade, please refer to packages/apps/Car/SystemUpdater/
This is an apk provided by Google on Android 9. It can be understood that it is an app for testing the update_engine upgrade on a local USB disk.

The main functions of this app are:
1. Read the upgrade file in the USB flash drive, and the user clicks on the target upgrade file.
2. Call updateEngine to pass the main parameters, and receive the callback of updateEngine to display the upgrade progress to the user.
3. After the upgrade is completed, notify powermanager to restart the machine.

The code directory is as follows (note, the following is the code for Android11):

├── src
│   └── com
│       └── android
│           └── car
│               └── systemupdater
│                   ├── DeviceListFragment.java
│                   ├── SystemUpdaterActivity.java
│                   ├── UpdateLayoutFragment.java
│                   ├── UpdateParser.java
│                   └── UpFragment.java
└── SystemUpdater.iml

1. User selects OTA upgrade package
   DeviceListFragment.java

/** Set the list of files shown on the screen. */
private void setFileList(List<File> files) {
    List<CarUiListItem> fileList = new ArrayList<>();
    for (File file : files) {
        CarUiContentListItem item = new CarUiContentListItem(CarUiContentListItem.Action.NONE);
        item.setTitle(file.getName());
        item.setOnItemClickedListener(i -> onFileSelected(file));
        fileList.add(item);
    }

    CarUiListItemAdapter adapter = new CarUiListItemAdapter(fileList);
    adapter.setMaxItems(CarUiRecyclerView.ItemCap.UNLIMITED);
    mFolderListView.setAdapter(adapter);
}

The click event is set here, and the following onFileSelected function will be called.

2. Determine whether it is an OTA upgrade package
   DeviceListFragment.java

/** Handle user selection of a file. */
private void onFileSelected(File file) {
    if (isUpdateFile(file)) {
        mFileStack.clear();
        mSystemUpdater.applyUpdate(file);//
    } else if (file.isDirectory()) {
        showFolderContent(file);
        mFileStack.push(file);
    } else {
        Toast.makeText(getContext(), R.string.invalid_file_type, Toast.LENGTH_LONG).show();
    }
}

If we select an upgrade file, the mSystemUpdater.applyUpdate(file) method will be called and the path to the upgrade file will be passed in.

3. Create a new UpdateLayoutFragment instance based on the passed in file object
   SystemUpdaterActivity.java

@Override
public void applyUpdate(File file) {
    UpdateLayoutFragment fragment = UpdateLayoutFragment.getInstance(file);//
    getSupportFragmentManager().beginTransaction()
            .replace(R.id.device_container, fragment, FRAGMENT_TAG)
            .addToBackStack(null)
            .commit();
}
4. UpdateLayoutFragmentmPackageVerifier.execute(mUpdateFile)
UpdateLayoutFragment.java
if (getArguments().getBoolean(EXTRA_RESUME_UPDATE)) {
    // Rejoin the update already in progress.
    showInstallationInProgress();
} else {
    // Extract the necessary information and begin the update.
    mPackageVerifier.execute(mUpdateFile);
}

If it is already updating, the progress information will be displayed. If it has not started yet, the mPackageVerifier.execute(mUpdateFile) method will be called.

5. Asynchronously execute update package verification and information extraction
   UpdateLayoutFragment.java

/** Attempt to verify the update and extract information needed for installation. */
private class UpdateVerifier extends AsyncTask<File, Void, UpdateParser.ParsedUpdate> {

    @Override
    protected UpdateParser.ParsedUpdate doInBackground(File... files) {
        Preconditions.checkArgument(files.length > 0, "No file specified");
        File file = files[0];
        try {
            return UpdateParser.parse(file);//UpdateParser的parse
        } catch (IOException e) {
            Log.e(TAG, String.format("For file %s", file), e);
            return null;
        }
    }

    @Override
    protected void onPostExecute(UpdateParser.ParsedUpdate result) {
        mProgressBar.setVisible(false);
        if (result == null) {
            showStatus(R.string.verify_failure);
            return;
        }
        if (!result.isValid()) {
            showStatus(R.string.verify_failure);
            Log.e(TAG, String.format("Failed verification %s", result));
            return;
        }
        if (Log.isLoggable(TAG, Log.INFO)) {
            Log.i(TAG, result.toString());
        }

        showInstallNow(result);//install now button
    }
}

6. There is an UpdateParser.parse(file) in the previous step 5. Let’s take a look.
   UpdateParser.java

/**
 * Parse a zip file containing a system update and return a non null ParsedUpdate.
 */
@Nullable
static ParsedUpdate parse(@NonNull File file) throws IOException {
    Preconditions.checkNotNull(file);

    long payloadOffset = 0;
    long payloadSize = 0;
    boolean payloadFound = false;
    String[] props = null;

    //zip payloadOffset、payloadSize、payloadFound props 。
    try (ZipFile zipFile = new ZipFile(file)) {
        Enumeration<? extends ZipEntry> entries = zipFile.entries();
        while (entries.hasMoreElements()) {
            ZipEntry entry = entries.nextElement();
            long fileSize = entry.getCompressedSize();
            if (!payloadFound) {
                payloadOffset += ZIP_FILE_HEADER + entry.getName().length();
                if (entry.getExtra() != null) {
                    payloadOffset += entry.getExtra().length;
                }
            }

            if (entry.isDirectory()) {
                continue;
            } else if (entry.getName().equals(PAYLOAD_BIN_FILE)) {// PAYLOAD_BIN_FILE 
                payloadSize = fileSize;
                payloadFound = true;
            } else if (entry.getName().equals(PAYLOAD_PROPERTIES)) {
                try (BufferedReader buffer = new BufferedReader(
                        new InputStreamReader(zipFile.getInputStream(entry)))) {
                    props = buffer.lines().toArray(String[]::new);
                }
            }
            if (!payloadFound) {
                payloadOffset += fileSize;
            }

            if (Log.isLoggable(TAG, Log.DEBUG)) {
                Log.d(TAG, String.format("Entry %s", entry.getName()));
            }
        }
    }
    return new ParsedUpdate(file, payloadOffset, payloadSize, props);
}

7. In the previous step 5, there is a showInstallNow(result); that is, the “Install Now” button is displayed and the content of the installation preparation interface is updated.
   UpdateLayoutFragment.java

/** Show the install now button. */
private void showInstallNow(UpdateParser.ParsedUpdate update) {
    mContentTitle.setText(R.string.install_ready);
    mContentInfo.append(getString(R.string.update_file_name, mUpdateFile.getName()));
    mContentInfo.append(System.getProperty("line.separator"));
    mContentInfo.append(getString(R.string.update_file_size));
    mContentInfo.append(Formatter.formatFileSize(getContext(), mUpdateFile.length()));
    mContentDetails.setText(null);
    MenuItem installButton = MenuItem.builder(getActivity())
            .setTitle(R.string.install_now)
            .setOnClickListener(i -> installUpdate(update))
            .build();
    mToolbar.setMenuItems(Collections.singletonList(installButton));
}

/** Attempt to install the update that is copied to the device. */
private void installUpdate(UpdateParser.ParsedUpdate parsedUpdate) {
    showInstallationInProgress();
    mUpdateEngine.applyPayload(
            parsedUpdate.mUrl, parsedUpdate.mOffset, parsedUpdate.mSize, parsedUpdate.mProps);
}

Here, use the applyPayload method of mUpdateEngine to start applying the update package.

The parameters of the applyPayload method are as follows:

• parsedUpdate.mUrl: URL address of the update package.
• parsedUpdate.mOffset: The offset of the updated package.
• parsedUpdate.mSize: The size of the update package.
• parsedUpdate.mProps: Update package properties, usually including signature, checksum and other information.

8. Let’s continue looking at the code of UpdateEngine.java

frameworks/base/core/java/android/os/UpdateEngine.java
    public UpdateEngine() {
        mUpdateEngine = IUpdateEngine.Stub.asInterface(
                ServiceManager.getService(UPDATE_ENGINE_SERVICE));}
    }

The constructor of the UpdateEngine class obtains UPDATE_ENGINE_SERVICE through ServiceManager, which is the update engine service registered in the ServiceManager at the C++ layer.

private IUpdateEngine mUpdateEngine;
.............................................
public void applyPayload(String url, long offset, long size, String[] headerKeyValuePairs) {
    try {
        mUpdateEngine.applyPayload(url, offset, size, headerKeyValuePairs);
    } catch (RemoteException e) {
        throw e.rethrowFromSystemServer();
    }
}

The applyPayload method actually calls the method of the IUpdateEngine interface, which is implemented by the update engine service of the C++ layer.

The timing diagram of the above process is as follows: