Plugin Directory

Changeset 3770239


Ignore:
Timestamp:
10/28/2026 08:48:12 AM (66 minutes ago)
Author:
andrewjbaker
Message:

Update trunk and assets to v3.2.860

Location:
cloudscale-backup-restore/trunk
Files:
7 edited

Legend:

Unmodified
Added
Removed
  • cloudscale-backup-restore/trunk/CHANGELOG.md

    r3754709 r3770239  
    11# Changelog
     2
     3## [3.2.860] - 2026-10-28
     4
     5### Changed
     6- **Cloud copies download in parallel when that is faster.** A copy from S3 or CloudScale Managed (restore,
     7  standby refresh, `wp csbr cloud-pull`) fetched one 128 MB range at a time, so a distant
     8  region was limited by the round trip, not the link, the same reason uploads went
     9  parallel in 3.2.834. After the first range, which says how big the backup is, each
     10  pass fetches up to four ranges at once with `Requests::request_multiple()`, streamed to
     11  disk. A pass never covers more than one old range, so it fits the same timeout and hop
     12  budget on a slow link. Each range must answer 206 with a Content-Range at the offset
     13  asked for and the exact size on disk; anything else is fetched again through the
     14  one-at-a-time path, and ranges are appended strictly in order, so an interrupted copy
     15  resumes as before. Behind a WP_Http proxy it stays one at a time.
     16  The first wave is timed against the first sequential range and the rest of the hop
     17  keeps whichever was faster: on 2026-10-28 a 3.55 GB Managed copy from Cape Town to the
     18  QA Pi on Wi-Fi took 524 s one range at a time and 695 s four at a time, byte-identical
     19  (`tools/benchmark-cloud-pull.sh`). Parallel pays where the round trip limits each
     20  connection, a distant region; where the link is full it only adds contention.
     21  `tests/s3-download-wave-test.php` (24 checks, wired into the build).
     22- **`wp csbr cloud-pull` resumes.** It used `s3_get()`, one streamed request that
     23  truncated on retry; it now uses the same resumable copy, keeping a shorter local file
     24  and discarding one that cannot be a prefix of the backup.
    225
    326## [3.2.858] - 2026-10-19
  • cloudscale-backup-restore/trunk/cloudscale-backup.php

    r3754709 r3770239  
    44 * Plugin URI:        https://cloudscale.consulting
    55 * Description:       No-nonsense WordPress backup and restore. Backs up database, media, plugins and themes into a single zip. Scheduled or manual, with safe restore and maintenance mode.
    6  * Version:     3.2.858
     6 * Version:     3.2.860
    77 * Author:            CloudScale
    88 * Author URI:        https://cloudscale.consulting
    … …  
    121121// wporg-risk-accepted-begin: PLUGINCHECK_SNIFF_47, WordPress.WP.I18n.TextDomainMismatch -- this plugin's WordPress.org slug is cloudscale-backup-restore and its Text Domain header says the same; existing installs keep the older folder name cloudscale-backup, which is what a folder-based check compares against
    122122
    123 define('CSBR_VERSION',    '3.2.858');
     123define('CSBR_VERSION',    '3.2.860');
    124124define('CSBR_AMI_POLL_MAX_AGE', 5 * 600);              // Stop polling after 5 attempts (50 minutes)
    125125define('CSBR_AMI_POLL_INTERVAL', 600);                 // Re-poll every 10 minutes
    … …  
    1033110331function csbr_s3_fetch_resumable( CSBR_AWS $client, string $bucket, string $key, string $dest, int $deadline = 0 ): array {
    1033210332    $chunk = defined( 'CSBR_PULL_CHUNK_BYTES' ) ? (int) CSBR_PULL_CHUNK_BYTES : ( 128 * 1024 * 1024 );
     10333    $lanes = defined( 'CSBR_PULL_LANES' ) ? max( 1, (int) CSBR_PULL_LANES ) : 4;
    1033310334    $have  = file_exists( $dest ) ? (int) filesize( $dest ) : 0;
    1033410335    $start = $have;
    1033510336    $total = 0;
    1033610337    $guard = 0;
     10338
     10339    // Bytes per second of the last full sequential range, and whether the first wave
     10340    // has been timed against it yet. See "KEEP WHICHEVER IS FASTER" below.
     10341    $seq_rate    = 0.0;
     10342    $wave_judged = false;
     10343
     10344    // Moves one finished range file onto the end of $dest and deletes it.
     10345    $append = static function ( string $part ) use ( $dest ): bool {
     10346        // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fopen -- appending the plugin's own partial download; WP_Filesystem has no append mode
     10347        $out = fopen( $dest, 'ab' );
     10348        // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fopen -- reading back the range just written
     10349        $in  = fopen( $part, 'rb' );
     10350        if ( false === $out || false === $in ) {
     10351            if ( is_resource( $out ) ) { fclose( $out ); } // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- paired with the fopen above
     10352            if ( is_resource( $in ) ) { fclose( $in ); } // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- paired with the fopen above
     10353            wp_delete_file( $part );
     10354            return false;
     10355        }
     10356        stream_copy_to_stream( $in, $out );
     10357        fclose( $in );  // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- paired with the fopen above
     10358        fclose( $out ); // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- paired with the fopen above
     10359        wp_delete_file( $part );
     10360        return true;
     10361    };
    1033710362
    1033810363    while ( true ) {
    … …  
    1034410369        }
    1034510370
    10346         $part = $dest . '.range';
    10347         $r    = $client->s3_get_range( $bucket, $key, $part, $have, $chunk );
    10348         if ( empty( $r['ok'] ) ) {
    10349             if ( file_exists( $part ) ) {
     10371        /*
     10372         * PARALLEL ONCE THE SIZE IS KNOWN.
     10373         *
     10374         * The first range is fetched alone, because its Content-Range is what says
     10375         * how big the object is. After that each pass fetches up to $lanes ranges at
     10376         * once, covering no more than one $chunk between them, so a pass is never
     10377         * longer than a sequential range was and the time budget below still holds.
     10378         * Ranges are appended strictly in order, so $dest only ever holds a correct
     10379         * prefix of the object and an interrupted copy resumes exactly as before.
     10380         */
     10381        $wave = ( $total > 0 && $lanes > 1 ) ? CSBR_AWS::download_wave( $have, $total, $chunk, $lanes ) : array();
     10382        if ( count( $wave ) > 1 ) {
     10383            $jobs = array();
     10384            foreach ( $wave as $i => $range ) {
     10385                $jobs[ $i ] = array( $range[0], $range[1], $dest . '.range' . $i );
     10386            }
     10387            $wave_from = $have;
     10388            $wave_t0   = microtime( true );
     10389            $landed    = $client->s3_get_ranges( $bucket, $key, $jobs );
     10390
     10391            foreach ( $jobs as $i => $job ) {
     10392                list( $offset, $length, $part ) = $job;
     10393                if ( empty( $landed[ $i ] ) ) {
     10394                    // A range the wave missed goes through the one-at-a-time path.
     10395                    $r = $client->s3_get_range( $bucket, $key, $part, $offset, $length );
     10396                    if ( empty( $r['ok'] ) || (int) ( $r['written'] ?? 0 ) !== $length ) {
     10397                        foreach ( $jobs as $left ) {
     10398                            if ( file_exists( $left[2] ) ) {
     10399                                wp_delete_file( $left[2] );
     10400                            }
     10401                        }
     10402                        $error = empty( $r['ok'] ) ? (string) ( $r['error'] ?? 'Ranged download failed.' ) : sprintf( 'S3 sent %d bytes for a %d-byte range.', (int) ( $r['written'] ?? 0 ), $length );
     10403                        return array( 'ok' => false, 'error' => $error, 'resumed_from' => $start );
     10404                    }
     10405                }
     10406                if ( ! $append( $part ) ) {
     10407                    foreach ( $jobs as $left ) {
     10408                        if ( file_exists( $left[2] ) ) {
     10409                            wp_delete_file( $left[2] );
     10410                        }
     10411                    }
     10412                    return array( 'ok' => false, 'error' => 'Could not append the downloaded range to ' . basename( $dest ) . '.' );
     10413                }
     10414                $have += $length;
     10415            }
     10416
     10417            if ( $have >= $total ) {
     10418                break;
     10419            }
     10420
     10421            /*
     10422             * KEEP WHICHEVER IS FASTER.
     10423             *
     10424             * Parallel ranges win when each connection is held back by the round
     10425             * trip, a distant region. When the link itself is the limit they only
     10426             * compete for it: measured on 2026-10-28, a 3.55 GB copy from Cape Town
     10427             * to a Pi on Wi-Fi took 524 s one range at a time and 695 s four at a
     10428             * time, the same bytes both ways. So the first wave is timed against the first range, and
     10429             * unless it is clearly faster the rest of this hop goes one at a time.
     10430             * One wave is the most this can cost; the next hop measures again.
     10431             */
     10432            if ( ! $wave_judged && $seq_rate > 0 ) {
     10433                $wave_judged = true;
     10434                $wave_rate   = ( $have - $wave_from ) / max( 0.001, microtime( true ) - $wave_t0 );
     10435                if ( $wave_rate < $seq_rate * 1.15 ) {
     10436                    $lanes = 1;
     10437                }
     10438                csbr_log( sprintf(
     10439                    '[CloudScale Backup & Restore] Cloud copy: %.1f MB/s one range at a time, %.1f MB/s in parallel, continuing %s.',
     10440                    $seq_rate / 1048576,
     10441                    $wave_rate / 1048576,
     10442                    1 === $lanes ? 'one range at a time' : 'in parallel'
     10443                ) );
     10444            }
     10445        } else {
     10446            $part   = $dest . '.range';
     10447            $seq_t0 = microtime( true );
     10448            $r      = $client->s3_get_range( $bucket, $key, $part, $have, $chunk );
     10449            if ( empty( $r['ok'] ) ) {
     10450                if ( file_exists( $part ) ) {
     10451                    wp_delete_file( $part );
     10452                }
     10453                // $dest is deliberately LEFT in place. It holds real bytes, and deleting
     10454                // them is what made every retry start from zero.
     10455                return array( 'ok' => false, 'error' => (string) ( $r['error'] ?? 'Ranged download failed.' ), 'resumed_from' => $start );
     10456            }
     10457
     10458            if ( 0 === $total && ! empty( $r['total'] ) ) {
     10459                $total = (int) $r['total'];
     10460            }
     10461
     10462            $written = (int) ( $r['written'] ?? 0 );
     10463            if ( $written > 0 ) {
     10464                if ( ! $append( $part ) ) {
     10465                    return array( 'ok' => false, 'error' => 'Could not append the downloaded range to ' . basename( $dest ) . '.' );
     10466                }
     10467                $have += $written;
     10468                // Timed to the same point as a wave, append included.
     10469                if ( $written === $chunk ) {
     10470                    $seq_rate = $chunk / max( 0.001, microtime( true ) - $seq_t0 );
     10471                }
     10472            } else {
    1035010473                wp_delete_file( $part );
    1035110474            }
    10352             // $dest is deliberately LEFT in place. It holds real bytes, and deleting
    10353             // them is what made every retry start from zero.
    10354             return array( 'ok' => false, 'error' => (string) ( $r['error'] ?? 'Ranged download failed.' ), 'resumed_from' => $start );
    10355         }
    10356 
    10357         if ( 0 === $total && ! empty( $r['total'] ) ) {
    10358             $total = (int) $r['total'];
    10359         }
    10360 
    10361         $written = (int) ( $r['written'] ?? 0 );
    10362         if ( $written > 0 ) {
    10363             // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fopen -- appending the plugin's own partial download; WP_Filesystem has no append mode
    10364             $out = fopen( $dest, 'ab' );
    10365             // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fopen -- reading back the range just written
    10366             $in  = fopen( $part, 'rb' );
    10367             if ( false === $out || false === $in ) {
    10368                 if ( is_resource( $out ) ) { fclose( $out ); } // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- paired with the fopen above
    10369                 if ( is_resource( $in ) ) { fclose( $in ); } // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- paired with the fopen above
    10370                 wp_delete_file( $part );
    10371                 return array( 'ok' => false, 'error' => 'Could not append the downloaded range to ' . basename( $dest ) . '.' );
     10475
     10476            if ( $written < $chunk ) {
     10477                // Short range means the end of the object.
     10478                break;
    1037210479            }
    10373             stream_copy_to_stream( $in, $out );
    10374             fclose( $in );  // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- paired with the fopen above
    10375             fclose( $out ); // phpcs:ignore WordPress.WP.AlternativeFunctions.file_system_operations_fclose -- paired with the fopen above
    10376             $have += $written;
    10377         }
    10378         wp_delete_file( $part );
    10379 
    10380         if ( $written < $chunk ) {
    10381             // Short range means the end of the object.
    10382             break;
    10383         }
    10384         if ( $total > 0 && $have >= $total ) {
    10385             break;
     10480            if ( $total > 0 && $have >= $total ) {
     10481                break;
     10482            }
    1038610483        }
    1038710484
  • cloudscale-backup-restore/trunk/includes/class-cloudscale-licence.php

    r3748367 r3770239  
    20062006                        <div style="margin-bottom:8px;font-size:0.75rem;color:#6b7280;">
    20072007                                <?php echo esc_html( ( $sub_provider ? $sub_provider . ' subscription:' : 'Subscription:' ) ); ?>
    2008                                 <code style="font-size:0.75rem;background:#f3f4f6;padding:2px 6px;border-radius:4px;"><?php echo esc_html( $sub_code ); ?></code>
     2008                                <code style="font-size:0.75rem;background:#f3f4f6;color:#1f2937;padding:2px 6px;border-radius:4px;"><?php echo esc_html( $sub_code ); ?></code>
    20092009                        </div>
    20102010                        <?php endif; ?>
    … …  
    20672067
    20682068                        <a href="#" id="<?php echo esc_attr( $card_id ); ?>-disconnect"
    2069                            style="display:block;text-align:center;font-size:0.75rem;color:#999;margin-top:4px;">
     2069                           style="display:block;text-align:center;font-size:0.75rem;color:#4b5563;margin-top:4px;">
    20702070                                <?php echo esc_html( 'Disconnect plugin only' ); ?>
    20712071                        </a>
  • cloudscale-backup-restore/trunk/includes/class-cloudscale-telegram.php

    r3754298 r3770239  
    911911        }
    912912
    913         private static function alert_origin(): string {
     913        /**
     914         * The origin block printed under every alert.
     915         *
     916         * @param bool $request_is_subject Whether the HTTP request executing this code is the alert's
     917         *        subject. True for almost every alert (a login, a firewall block: the request is the
     918         *        event). False when the current request only CARRIES the alert, which it never caused,
     919         *        for example the Edge Gateway POSTing a batch of events it detected elsewhere: there the
     920         *        courier is a trusted local request to our own events route, and 'From'/'Request' would
     921         *        name that courier as though it were the attacker. The real actor is then in the body,
     922         *        so only the host is printed and the request lines are left off.
     923         * @return string
     924         */
     925        private static function alert_origin( bool $request_is_subject = true ): string {
    914926                $host = self::host_label();
    915927
    … …  
    932944                $lines   = array();
    933945                $lines[] = 'Host: ' . $host . ( '' !== $server_ip ? ' (container ' . $server_ip . ')' : '' );
     946
     947                // The request lines describe the request in flight. When that request is only the courier
     948                // of an event detected elsewhere, they name the courier, not the attacker, so they are
     949                // omitted and the body's own 'from X' stands as the source.
     950                if ( ! $request_is_subject ) {
     951                        return "\n" . implode( "\n", $lines );
     952                }
     953
    934954                $lines[] = 'From: ' . self::trigger_source();
    935955
    … …  
    11591179         * @param string $source  Plugin/feature name shown in the header (e.g. "Backup and Restore").
    11601180         * @param string $level   Alert level: info, warning, error, critical.
    1161          */
    1162         public static function send( string $text, string $source = '', string $level = 'info' ): bool {
     1181         * @param bool   $request_is_subject Whether the HTTP request in flight is this alert's subject.
     1182         *        Defaults true. Pass false for an alert the current request merely carries (an Edge
     1183         *        Gateway event batch), so the origin block does not name the courier as the actor.
     1184         */
     1185        public static function send( string $text, string $source = '', string $level = 'info', bool $request_is_subject = true ): bool {
    11631186                // Before anything else, and before any call site's own toggle: a muted install
    11641187                // sends nothing at all. See OPTION_MUTED for why this belongs here and not at
    … …  
    12021225                $full = self::alert_prefix( $source, $level )
    12031226                        . self::localise_utc_stamps( $text )
    1204                         . "\n" . self::alert_origin()
     1227                        . "\n" . self::alert_origin( $request_is_subject )
    12051228                        . "\n\n" . self::local_time();
    12061229
  • cloudscale-backup-restore/trunk/includes/class-csbr-aws.php

    r3754649 r3770239  
    155155     */
    156156    private const MULTIPART_LANES = 4;
     157
     158    /**
     159     * Smallest range a download wave splits a stretch into, see download_wave().
     160     *
     161     * @since 3.2.860
     162     */
     163    private const DOWNLOAD_MIN_RANGE = 8 * 1024 * 1024;
    157164
    158165    /**
    … …  
    12111218    }
    12121219
     1220    /**
     1221     * Split the next stretch of a download into ranges to fetch at once.
     1222     *
     1223     * The stretch is never more than $chunk, the size one sequential range already
     1224     * was, so a wave moves no more bytes than a single range did and fits inside the
     1225     * same timeout and hop budget even when the link is slow. It only ever gets there
     1226     * sooner: a distant region limits each connection by its round trip, not by the
     1227     * link, which is what made uploads four times slower than they needed to be
     1228     * before 3.2.834.
     1229     *
     1230     * A stretch too small to split usefully stays one range, so a nearly finished
     1231     * download does not open four connections for a few megabytes.
     1232     *
     1233     * Pure, so tests/s3-download-wave-test.php can drive it.
     1234     *
     1235     * @since 3.2.860
     1236     * @param int $have  Bytes already on disk, the offset of the first range.
     1237     * @param int $total Size of the whole object.
     1238     * @param int $chunk Most bytes one wave may cover.
     1239     * @param int $lanes Most ranges in one wave.
     1240     * @return array<int, array{0:int,1:int}> Offset and length of each range, in order.
     1241     */
     1242    public static function download_wave( int $have, int $total, int $chunk, int $lanes = self::MULTIPART_LANES ): array {
     1243        if ( $have < 0 || $total <= $have || $chunk <= 0 ) {
     1244            return array();
     1245        }
     1246        $span  = min( $chunk, $total - $have );
     1247        $count = max( 1, min( $lanes, intdiv( $span, self::DOWNLOAD_MIN_RANGE ) ) );
     1248        $size  = (int) ceil( $span / $count );
     1249
     1250        $wave   = array();
     1251        $offset = $have;
     1252        $end    = $have + $span;
     1253        while ( $offset < $end ) {
     1254            $length  = min( $size, $end - $offset );
     1255            $wave[]  = array( $offset, $length );
     1256            $offset += $length;
     1257        }
     1258        return $wave;
     1259    }
     1260
     1261    /**
     1262     * Fetch several byte ranges of an object at once, each into its own file.
     1263     *
     1264     * Returns only the ranges S3 answered in full: 206, a Content-Range starting at
     1265     * the offset asked for, and exactly the bytes asked for on disk. Anything else is
     1266     * deleted and left out, and the caller fetches that range again through
     1267     * s3_get_range(), the one-at-a-time path. So, like s3_multipart_upload_wave(), a
     1268     * wave can only make a download faster: at worst it returns nothing.
     1269     *
     1270     * Same transport as the upload wave, for the same reason: WordPress has no
     1271     * parallel wrapper and direct cURL is not allowed on WordPress.org. A site behind
     1272     * a proxy that WP_Http would have applied gets the one-at-a-time path. Each body
     1273     * streams to disk through 'filename', so a wave holds no range in memory.
     1274     *
     1275     * @since 3.2.860
     1276     * @param string                                  $bucket  Bucket name.
     1277     * @param string                                  $key     Object key.
     1278     * @param array<int, array{0:int,1:int,2:string}> $ranges  Id => offset, length, destination file.
     1279     * @param int                                     $timeout Seconds for each request.
     1280     * @return array<int, true> Ids of the ranges that landed whole.
     1281     */
     1282    public function s3_get_ranges( string $bucket, string $key, array $ranges, int $timeout = 600 ): array {
     1283        if ( ! $ranges || ! class_exists( '\WpOrg\Requests\Requests' ) ) {
     1284            return array();
     1285        }
     1286        if ( class_exists( 'WP_HTTP_Proxy' ) && ( new WP_HTTP_Proxy() )->is_enabled() ) {
     1287            return array();
     1288        }
     1289
     1290        $encoded_key = implode( '/', array_map( 'rawurlencode', explode( '/', $key ) ) );
     1291        // wporg-risk-accepted: CS_REJECT_19 -- the AWS S3 and EC2 API endpoint the administrator's own bucket or instance lives on, documented under External services; nothing is offloaded, this is the storage the user chose
     1292        // phpcs:ignore PluginCheck.CodeAnalysis.Offloading.OffloadedContent -- S3 REST endpoint for the user's OWN bucket, to retrieve their own backups. Nothing is served to a browser.
     1293        $url = 'https://' . $bucket . '.s3.' . $this->region . '.amazonaws.com/' . ltrim( $encoded_key, '/' );
     1294
     1295        $requests = array();
     1296        foreach ( $ranges as $id => $range ) {
     1297            list( $offset, $length, $dest ) = $range;
     1298            // The Range header stays unsigned, as in s3_get_range().
     1299            $headers          = $this->sign( 'GET', $url, array(), '', 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855' );
     1300            $headers['Range'] = 'bytes=' . $offset . '-' . ( $offset + $length - 1 );
     1301
     1302            $requests[ $id ] = array(
     1303                'url'     => $url,
     1304                'type'    => 'GET',
     1305                'headers' => $headers,
     1306                'data'    => array(),
     1307                'options' => array(
     1308                    'timeout'         => $timeout,
     1309                    'connect_timeout' => self::CONNECT_TIMEOUT,
     1310                    'verify'          => CSBR_Paths::root_file( WPINC . '/certificates/ca-bundle.crt' ),
     1311                    'filename'        => $dest,
     1312                ),
     1313            );
     1314        }
     1315
     1316        try {
     1317            $responses = \WpOrg\Requests\Requests::request_multiple( $requests );
     1318        } catch ( \Throwable $e ) {
     1319            if ( function_exists( 'csbr_log' ) ) {
     1320                csbr_log( '[CloudScale Backup & Restore] Parallel S3 ranges failed to start, continuing one range at a time: ' . $e->getMessage() );
     1321            }
     1322            $responses = array();
     1323        }
     1324
     1325        $landed = array();
     1326        foreach ( $ranges as $id => $range ) {
     1327            list( $offset, $length, $dest ) = $range;
     1328            $response = $responses[ $id ] ?? null;
     1329
     1330            $ok = $response instanceof \WpOrg\Requests\Response && 206 === (int) $response->status_code;
     1331            if ( $ok ) {
     1332                $cr = (string) ( $response->headers['content-range'] ?? '' );
     1333                $ok = 1 === preg_match( '#^bytes (\d+)-#', $cr, $m ) && (int) $m[1] === $offset;
     1334            }
     1335            if ( $ok ) {
     1336                clearstatcache( true, $dest );
     1337                $ok = file_exists( $dest ) && (int) filesize( $dest ) === $length;
     1338            }
     1339
     1340            if ( $ok ) {
     1341                $landed[ $id ] = true;
     1342            } elseif ( file_exists( $dest ) ) {
     1343                wp_delete_file( $dest );
     1344            }
     1345        }
     1346
     1347        $missed = count( $ranges ) - count( $landed );
     1348        if ( $missed > 0 && function_exists( 'csbr_log' ) ) {
     1349            csbr_log( sprintf(
     1350                '[CloudScale Backup & Restore] %d of %d parallel S3 ranges did not land, fetching them again one at a time.',
     1351                $missed,
     1352                count( $ranges )
     1353            ) );
     1354        }
     1355
     1356        unset( $responses, $requests );
     1357        gc_collect_cycles();
     1358
     1359        return $landed;
     1360    }
     1361
    12131362    public function s3_get( string $bucket, string $key, string $dest_path ): array {
    12141363        $encoded_key    = implode( '/', array_map( 'rawurlencode', explode( '/', $key ) ) );
  • cloudscale-backup-restore/trunk/includes/class-csbr-cli.php

    r3754649 r3770239  
    580580                $key = rtrim( $this->prefix(), '/' ) . '/' . $filename;
    581581                WP_CLI::log( 'Downloading s3://' . $this->bucket() . '/' . $key . ' (' . CSBR_Backup_Utils::format_size( $expected ) . ')...' );
    582                 $result = $s3->s3_get( $this->bucket(), $key, $local );
     582
     583                // The same ranged, parallel and resumable copy the admin screen and the
     584                // automatic restore use. It appends to what is on disk, so a shorter local
     585                // file is resumed, and one that cannot be a prefix of this backup goes first.
     586                if ( file_exists( $local ) && ( $force || (int) filesize( $local ) > $expected ) ) {
     587                        wp_delete_file( $local );
     588                }
     589                $result = csbr_s3_fetch_resumable( $s3, $this->bucket(), $key, $local );
    583590                if ( empty( $result['ok'] ) || ! file_exists( $local ) ) {
    584591                        WP_CLI::error( 'Download failed: ' . ( $result['error'] ?? 'file missing after download' ) );
  • cloudscale-backup-restore/trunk/readme.txt

    r3754709 r3770239  
    55Tested up to: 7.1
    66Requires PHP: 8.1
    7 Stable tag: 3.2.858
     7Stable tag: 3.2.860
    88License: GPL-2.0-or-later
    99License URI: https://www.gnu.org/licenses/gpl-2.0.html
    … …  
    2020* **Built for large sites.** The database is streamed in 500-row pages, never loaded into memory. Uploads go in chunks: S3 multipart over 100 MB, and OneDrive, Dropbox and Google Drive chunks are retried up to 5 times on a dropped connection, throttle or server error, resuming where the provider says it stopped.
    2121* **Five destinations at once.** Amazon S3, Google Drive, Dropbox and Microsoft OneDrive on your own accounts, plus CloudScale Managed Cloud. Every destination you enable gets every backup.
    22 * **Regional affinity.** Managed Cloud stores backups in one of ten regions: US East, US West, South America, Ireland, Frankfurt, Cape Town, Mumbai, Singapore, Tokyo or Sydney. Automatic mode measures which is nearest your site; or pick one for data residency.
    23 * **Automated recovery to a new site.** A standby server restores itself from the newest cloud backup on a schedule, refuses a stale one, and reports recovery time and a file-by-file reconciliation.
     22* **Ten storage regions on six continents.** Managed Cloud keeps backups on Amazon S3 in US East, US West, São Paulo, Ireland, Frankfurt, Cape Town, Mumbai, Singapore, Tokyo and Sydney. Automatic mode times each region from your server and keeps your backups in the nearest one, so copies and restores take the shortest, lowest-latency path. Large backups upload up to four parts at a time, downloads go parallel whenever that is faster, and a restore starts at once, with no archive thaw. Or pick a region for data residency.
     23* **Automated disaster recovery.** A standby server restores itself from the newest cloud backup on a schedule, refuses a stale one, and reports recovery time and a file-by-file reconciliation.
    2424* **Self-healing backups.** A backup killed mid-run (deploy, PHP restart, out of memory) is detected through a file lock and finished or restarted, and you are told. Every archive can be integrity-checked before it is uploaded or older backups are deleted.
    2525* **Rollback for bad plugin updates.** Automatic Crash Recovery copies a plugin before WordPress updates it and can roll it back if the site starts failing.
    … …  
    4141* **Full restore** puts back the backup's files, then its database. It runs on the server, so you can close the page
    4242* **Specific tables only** or a **single post or page** with its metadata, no maintenance mode needed
    43 * Restore from this server, from an uploaded `.zip` or `.sql`, or pull straight from S3, Google Drive, Dropbox, OneDrive or Managed Cloud to the server. S3 and Managed Cloud downloads resume where they stopped
     43* Restore from this server, from an uploaded `.zip` or `.sql`, or pull straight from S3, Google Drive, Dropbox, OneDrive or Managed Cloud to the server. S3 and Managed Cloud downloads go four ranges at a time when that is faster, and resume where they stopped
    4444* Encrypted backups are checked against your password before anything is touched
    4545* Visitors see a maintenance page; it is removed whether the restore succeeds or fails
    … …  
    7171= Managed Cloud =
    7272
    73 Don't have a cloud account? CloudScale Managed Cloud is $5/month for 750 GB on Amazon S3 Glacier Instant Retrieval, in the region you choose or the nearest one. Your own S3, Google Drive, Dropbox and OneDrive destinations are always free.
     73Don't have a cloud account? CloudScale Managed Cloud is $5/month for 750 GB on Amazon S3 Glacier Instant Retrieval, in ten regions worldwide: the one nearest your site by default, or the one you choose. Instant Retrieval means a restore downloads straight away, with no archive thaw. Your own S3, Google Drive, Dropbox and OneDrive destinations are always free.
    7474
    7575**Automatic backups are off by default.** Run your first backup by hand to confirm your server, then switch on a schedule. Full documentation: [help.cloudscale.consulting](https://help.cloudscale.consulting/plugin-help/backup-restore/).
    … …  
    216216== Changelog ==
    217217
     218= 3.2.860 =
     219* IMPROVED: Copies from Amazon S3 and CloudScale Managed (restores, standby refreshes, `wp csbr cloud-pull`) download four ranges at a time when that is faster. Each copy times both ways and keeps the faster, since on a link that is already full four connections are slower. Each range is checked before it is added, and an interrupted copy still resumes where it stopped.
     220* IMPROVED: `wp csbr cloud-pull` resumes a partial download instead of starting again.
     221
    218222= 3.2.858 =
    219223* FIX: Uninstall now removes the plugin's state files in wp-content. Backup archives, the token naming their folder, and the site-role and alert-label markers shared with other CloudScale plugins are kept on purpose.
Note: See TracChangeset for help on using the changeset viewer.