2017年2月2日 星期四

透過 PHP 控制檔案下載

有時,某些檔案下載必需管制對象,此時需透過 PHP 來控制檔案下載。
下面整理了幾個作法,可參考自行試做看看,整合成自己需要的。

在 PHP 的官網上,提供一個最簡單的作法,
 <?php
$file 
'monkey.gif';

if (
file_exists($file)) {
    
header('Content-Description: File Transfer');
    
header('Content-Type: application/octet-stream');
    
header('Content-Disposition: attachment; filename="'.basename($file).'"');
    
header('Expires: 0');
    
header('Cache-Control: must-revalidate');
    
header('Pragma: public');
    
header('Content-Length: ' filesize($file));
    
readfile($file);
    exit;
}


注意,filename 的 「"」不能少掉,因為檔名可能包含空白。

另外,參考下述網頁的討論,
Fastest Way to Serve a File Using PHP

可用 X-SendFile header,交由 Apache 下載,對於大檔案,會比較有效率。

header("X-Sendfile: $file_name");
header("Content-type: application/octet-stream");
header('Content-Disposition: attachment; filename="' . basename($file_name) . '"');
Where $file_name is the full path on the file system.

再來,可以控制下載速率,支援續傳功能
function Download($path, $speed = null, $multipart = true)
{
    while (ob_get_level() > 0)
    {
        ob_end_clean();
    }

    if (is_file($path = realpath($path)) === true)
    {
        $file = @fopen($path, 'rb');
        $size = sprintf('%u', filesize($path));
        $speed = (empty($speed) === true) ? 1024 : floatval($speed);

        if (is_resource($file) === true)
        {
            set_time_limit(0);

            if (strlen(session_id()) > 0)
            {
                session_write_close();
            }

            if ($multipart === true)
            {
                $range = array(0, $size - 1);

                if (array_key_exists('HTTP_RANGE', $_SERVER) === true)
                {
                    $range = array_map('intval', explode('-', preg_replace('~.*=([^,]*).*~', '$1', $_SERVER['HTTP_RANGE'])));

                    if (empty($range[1]) === true)
                    {
                        $range[1] = $size - 1;
                    }

                    foreach ($range as $key => $value)
                    {
                        $range[$key] = max(0, min($value, $size - 1));
                    }

                    if (($range[0] > 0) || ($range[1] < ($size - 1)))
                    {
                        header(sprintf('%s %03u %s', 'HTTP/1.1', 206, 'Partial Content'), true, 206);
                    }
                }

                header('Accept-Ranges: bytes');
                header('Content-Range: bytes ' . sprintf('%u-%u/%u', $range[0], $range[1], $size));
            }

            else
            {
                $range = array(0, $size - 1);
            }

            header('Pragma: public');
            header('Cache-Control: public, no-cache');
            header('Content-Type: application/octet-stream');
            header('Content-Length: ' . sprintf('%u', $range[1] - $range[0] + 1));
            header('Content-Disposition: attachment; filename="' . basename($path) . '"');
            header('Content-Transfer-Encoding: binary');

            if ($range[0] > 0)
            {
                fseek($file, $range[0]);
            }

            while ((feof($file) !== true) && (connection_status() === CONNECTION_NORMAL))
            {
                echo fread($file, round($speed * 1024)); flush(); sleep(1);
            }

            fclose($file);
        }

        exit();
    }

    else
    {
        header(sprintf('%s %03u %s', 'HTTP/1.1', 404, 'Not Found'), true, 404);
    }

    return false;
}
The code is as efficient as it can be, it closes the session handler so that other PHP scripts can run concurrently for the same user / session. It also supports serving downloads in ranges (which is also what Apache does by default I suspect), so that people can pause/resume downloads and also benefit from higher download speeds with download accelerators. It also allows you to specify the maximum speed (in Kbps) at which the download (part) should be served via the $speedargument.

再一個,

A better implementation, with cache support, customized http headers.
serveStaticFile($fn, array(
        'headers'=>array(
            'Content-Type' => 'image/x-icon',
            'Cache-Control' =>  'public, max-age=604800',
            'Expires' => gmdate("D, d M Y H:i:s", time() + 30 * 86400) . " GMT",
        )
    ));

function serveStaticFile($path, $options = array()) {
    $path = realpath($path);
    if (is_file($path)) {
        if(session_id())
            session_write_close();

        header_remove();
        set_time_limit(0);
        $size = filesize($path);
        $lastModifiedTime = filemtime($path);
        $fp = @fopen($path, 'rb');
        $range = array(0, $size - 1);

        header('Last-Modified: ' . gmdate("D, d M Y H:i:s", $lastModifiedTime)." GMT");
        if (( ! empty($_SERVER['HTTP_IF_MODIFIED_SINCE']) && strtotime($_SERVER['HTTP_IF_MODIFIED_SINCE']) == $lastModifiedTime ) ) {
            header("HTTP/1.1 304 Not Modified", true, 304);
            return true;
        }

        if (isset($_SERVER['HTTP_RANGE'])) {
            //$valid = preg_match('^bytes=\d*-\d*(,\d*-\d*)*$', $_SERVER['HTTP_RANGE']);
            if(substr($_SERVER['HTTP_RANGE'], 0, 6) != 'bytes=') {
                header('HTTP/1.1 416 Requested Range Not Satisfiable', true, 416);
                header('Content-Range: bytes */' . $size); // Required in 416.
                return false;
            }

            $ranges = explode(',', substr($_SERVER['HTTP_RANGE'], 6));
            $range = explode('-', $ranges[0]); // to do: only support the first range now.

            if ($range[0] === '') $range[0] = 0;
            if ($range[1] === '') $range[1] = $size - 1;

            if (($range[0] >= 0) && ($range[1] <= $size - 1) && ($range[0] <= $range[1])) {
                header('HTTP/1.1 206 Partial Content', true, 206);
                header('Content-Range: bytes ' . sprintf('%u-%u/%u', $range[0], $range[1], $size));
            }
            else {
                header('HTTP/1.1 416 Requested Range Not Satisfiable', true, 416);
                header('Content-Range: bytes */' . $size);
                return false;
            }
        }

        $contentLength = $range[1] - $range[0] + 1;

        //header('Content-Disposition: attachment; filename="xxxxx"');
        $headers = array(
            'Accept-Ranges' => 'bytes',
            'Content-Length' => $contentLength,
            'Content-Type' => 'application/octet-stream',
        );

        if(!empty($options['headers'])) {
            $headers = array_merge($headers, $options['headers']);
        }
        foreach($headers as $k=>$v) {
            header("$k: $v", true);
        }

        if ($range[0] > 0) {
            fseek($fp, $range[0]);
        }
        $sentSize = 0;
        while (!feof($fp) && (connection_status() === CONNECTION_NORMAL)) {
            $readingSize = $contentLength - $sentSize;
            $readingSize = min($readingSize, 512 * 1024);
            if($readingSize <= 0) break;

            $data = fread($fp, $readingSize);
            if(!$data) break;
            $sentSize += strlen($data);
            echo $data;
            flush();
        }

        fclose($fp);
        return true;
    }
    else {
        header('HTTP/1.1 404 Not Found', true, 404);
        return false;
    }
}

結論
If you wish to hide where the file is located and people with specific privilege may download the file then it is a good idea to use PHP as relay, and you have to sacrifice some CPU time to gain more security and control.
 

續傳 (Resume download) 在 Apache 的 X-SendFile 與 FireFox 的問題  

2018-02-20 補記
在續傳 (Resume download) 時,瀏覽器除了會送出 HTTP_RANGE 的 header 外,還會送出其他的 header。如 Firefox 會額外送出 HTTP_IF_MATCH 和 HTTP_IF_UNMODIFIED_SINCE,使得 Apache 傳回 404 的錯誤。而 Chrome 會多送出 HTTP_IF_RANGE,但這不影響正常運作。不論如何,把這些多的 header 都拿掉,在 Apache 的設定檔,或 .htaccess 中,加入下面的設定。

# 將會造成 resume download 的 header 拿掉
SetEnvIf Range .+ HAS_RANGE_HEADER
RequestHeader unset If-Range env=HAS_RANGE_HEADER
RequestHeader unset If-Match env=HAS_RANGE_HEADER
RequestHeader unset If-Unmodified-Since env=HAS_RANGE_HEADER

2017年1月22日 星期日

小試 PHP 5.6 vs PHP 7.0 vs PHP 7.1

最近想昇級到 PHP 7,本以為最新版的 PHP 7.1 會是最快的,但看到網路上的比較,發現最快的是 PHP 7.0。看來,是不用追最新的版本啊。下面,自己來做個小測試吧。

在 Gentoo 下,使用 elselect 來切換不同版本的 PHP,下面的指令為切換成 PHP 7.0
# eselect php set cli php7.0

使用 PHP 內附的 httpd server
 $ php -S 127.0.0.1:8080

使用 siege 來測試效能
$ siege -c 10 -r 10 -u http://127.0.0.1:8080/ntu-ocw/index.php/ocw/cou/104S113

網頁程式,是自己管的開放式課程 (NTU OCW),使用 Laravel 5.1。

三者比較的結果,PHP 7.0 > PHP 7.1 > PHP 5.6,和網上看到的結果一致。

看起來有夠少的,可是測一下 index.html,卻也只有 Transaction rate: 14.27 trans/sec,實在是不可思議啊,是我不會用 siege?

補充說明

本文看看就好,我必須承認沒有什麼參考性。此網站使用 Laravel framework,受到 session 的影響很大,假如不啟動 session,是可以讓效能提升很多的。

PHP 7.1.1 的結果

Transactions:                 100 hits
Availability:              100.00 %
Elapsed time:               10.77 secs
Data transferred:            5.63 MB
Response time:                0.39 secs
Transaction rate:            9.29 trans/sec
Throughput:                0.52 MB/sec
Concurrency:                3.63
Successful transactions:         100
Failed transactions:               0
Longest transaction:            0.76
Shortest transaction:            0.09

PHP 7.0.15 的結果

Transactions:                 100 hits
Availability:              100.00 %
Elapsed time:                9.83 secs
Data transferred:            5.63 MB
Response time:                0.35 secs
Transaction rate:           10.17 trans/sec
Throughput:                0.57 MB/sec
Concurrency:                3.54
Successful transactions:         100
Failed transactions:               0
Longest transaction:            0.61
Shortest transaction:            0.08

PHP 5.6.29 的結果

Transactions:                 100 hits
Availability:              100.00 %
Elapsed time:               11.37 secs
Data transferred:            5.63 MB
Response time:                0.42 secs
Transaction rate:            8.80 trans/sec
Throughput:                0.50 MB/sec
Concurrency:                3.68
Successful transactions:         100
Failed transactions:               0
Longest transaction:            0.78
Shortest transaction:            0.10

正式網站 Apache 2.2.15 + PHP 5.4.42

Transactions:                 100 hits
Availability:              100.00 %
Elapsed time:                8.15 secs
Data transferred:            5.74 MB
Response time:                0.22 secs
Transaction rate:           12.27 trans/sec
Throughput:                0.70 MB/sec
Concurrency:                2.69
Successful transactions:         100
Failed transactions:               0
Longest transaction:            0.40
Shortest transaction:            0.15

nginx/1.10.2  +  PHP/7.0.14

Transactions:                 100 hits
Availability:              100.00 %
Elapsed time:                8.62 secs
Data transferred:            5.95 MB
Response time:                0.18 secs
Transaction rate:           11.60 trans/sec
Throughput:                0.69 MB/sec
Concurrency:                2.05
Successful transactions:         100
Failed transactions:               0
Longest transaction:            0.30
Shortest transaction:            0.13

2017年1月20日 星期五

ReiserFS 與 Gentoo

Benchmark
http://marc.info/?l=reiserfs-devel&m=121484256609180&w=2 ( 2008-06-30)

mkfs:
ext3-1: 0,02s user 0,99s system 6% cpu 14,816 total
ext3-2: 0,03s user 3,63s system 9% cpu 39,030 total
jfs-1: 0,00s user 0,04s system 11% cpu 0,337 total
jfs-2: 0,00s user 0,09s system 12% cpu 0,731 total
xfs-1; 0,00s user 0,01s system 2% cpu 0,322 total
xfs-2: 0,00s user 0,01s system 0% cpu 0,907 total
reiserfs-1: 0,01s user 0,04s system 0% cpu 10,455 total
reiserfs-2: 0,01s user 0,07s system 2% cpu 3,502 total
reiser4-1: 0,00s user 0,01s system 2% cpu 0,423 total
reiser4-2:0,01s user 0,02s system 1% cpu 2,619 total

disk usage
ext3: 48062440  22160100  23460864  49%
jfs: 48795072  22049220  26745852  46%
xfs: 48805696  21979288  26826408  46%
reiserfs: 48828008  21960572  26867436  45%
reiser4: 46397568  21190652  25206916  46%

create
ext3: 1,08s user 105,19s system 3% cpu 45:17,78 total
jfs: 0,79s user 64,93s system 6% cpu 16:14,70 total
xfs: 1,09s user 82,65s system 2% cpu 1:04:30,78 total
reiserfs: 1,12s user 183,73s system 7% cpu 41:17,34 total
reiser4: 0,88s user 123,96s system 15% cpu 13:04,77 total

copy
ext3: 1,09s user 114,32s system 3% cpu 50:15,30 total
jfs: 0,84s user 68,99s system 6% cpu 17:22,76 total
xfs: 1,10s user 90,33s system 3% cpu 40:03,23 total
reiserfs: 1,03s user 173,93s system 11% cpu 25:56,22 total
reiser4: 0,89s user 142,65s system 17% cpu 14:01,58 total

move
ext3: 0,00s user 0,01s system 4% cpu 0,218 total
jfs: 0,00s user 0,00s system 0% cpu 0,353 total
xfs: 0,00s user 0,01s system 2% cpu 0,540 total
reiserfs: 0,00s user 0,01s system 0% cpu 0,688 total
reiser4: 0,00s user 0,01s system 1% cpu 0,602 total

rem
ext3-1: 0,04s user 5,12s system 6% cpu 1:17,75 total
ext3-1: 0,04s user 5,74s system 4% cpu 2:05,16 total
jfs-1: 0,05s user 4,74s system 3% cpu 2:26,86 total
jfs-2: 0,05s user 4,55s system 3% cpu 2:27,94 total 
xfs-1: 0,04s user 10,76s system 1% cpu 12:13,20 total
xfs-2: 0,04s user 11,23s system 1% cpu 13:24,95 total
reiserfs-1: 0,04s user 16,59s system 30% cpu 53,700 total
reiserfs-2: 0,06s user 16,57s system 29% cpu 56,769 total
reiser4-1: 0,07s user 23,64s system 22% cpu 1:47,22 total
reiser4-2: 0,06s user 20,18s system 19% cpu 1:41,35 total

原測試者評語:
I was very surprised by jfs and xfs. The first was faster than expected (even with the unfairness in its favour) and xfs was much, much slower than expected. XFS was pretty fast with the films, but suffered a lot with the emails, while reiserfs and reiser4 dealt very well with the emails.

註: JFS 看起來,雖然不錯,但有crash的風險。ReiserFS 和 Reiser4 的優點,在於處理小檔案。但是 ReiserFS 不支援 SSD 的 discard 動作,而 Reiser4 沒有進 Linux 的 kernel,看起來,將來也不可能被採用。未來之星是 Btrfs。

結論,別再關注 ReiserFS,不值得的。

Reiser4 Gentoo FAQ
https://forums.gentoo.org/viewtopic-t-706171.html

2017年1月18日 星期三

Laravel 的 Implicit controller

從 CodeIgniter (CI) 開始使用 PHP 的 MVC framework,然後是 Laravel 3.1,只要建好 controller,就自動有連結可用。再跳到 Laravel 5.0,變成要在 route.php 裡宣告 Route::controllers(),這些都是所謂的 Implicit controller。

這個 Implicit controller,在 Laravel 5.2 時,變成 deprecated,在後面的版本,這個就會被取消。

為何要取消,心中有一些疑惑,也許有很多英文的討論吧,但沒能力看那些討論。但自己想偷懶,也沒有寫成符合 RESTful 的程式,非常想要 Implicit controller 的功能。後來,找到了這篇討論
https://laravel-china.org/topics/3614

有空再仔細看一下相關討論,思考一下該怎麼做。也許,開始時,會直接加上 Implicit controller 的作法吧,終究是偷懶的。

網誌存檔