From patchwork Mon Aug 13 17:20:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 144036 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp3334562ljj; Mon, 13 Aug 2018 10:21:05 -0700 (PDT) X-Google-Smtp-Source: AA+uWPyxVxwgvAiS6+U4ZEyM8MJ/2sUnIAIcAkosSWaryo3Th8/GYFcsTtqxvZEZinyqXySpGrVn X-Received: by 2002:a17:902:286a:: with SMTP id e97-v6mr2300243plb.340.1534180865539; Mon, 13 Aug 2018 10:21:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534180865; cv=none; d=google.com; s=arc-20160816; b=YQ1MtUOIVL9jr9Li7j4az4TMLrwl+Ud620B4YDm1+Ap1G0Tuwh4v9Gtx8oo9xIcepG QU84DxNdzLBDvxHn+kewWpfoRRtsRztd6AIm0uWaK4tXDhZYAi3bGd+yHg/S3V13PoAo OuOyIidPFdn9Qac02aaRr/RxwHAVR2EK2IUVkdlTiS6uDB1KkjXPecr9UcIB9sOibPvz rRYooU+9jjutCpZRN+cGIspmyft4JA7XR1/87kxa3w7sQ8wX6Zo79C5A5hhXK1lZl2m+ 9wNsBLOMJKih278KGtW9dFUP6RL4zY8nipTtjMegAzJ0+RacLn3IUPJ2dh/s8K4ic2Hn u5XA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=my02wVF9pzfB5zMmQdvjXa+jlAZPE84LMCKP26lg2aM=; b=hUIyf/f7VzkpiWrGvmRNvSkHijF700P70hYhoeVHXyDWgT1C0mG9tSZ1SL9EMmISqx llrE8Kkb+Y6iRapEnlRzu7HVO4DpNQwprbxhGZ38ACBHY0trNudzqNAGj3Lpxfk93RXc Dgg4nd1RBEBPpQzY7LY5IhhE9dZ9Hcd8VvyEl6YBNLTq39lFgpDr71Sy/qxP7L2m/r7N jxkIiq38ZbsB7L/y52rhCeIVLlqG/7d1xHjGshQDk2sjUa7t6sFtkGrapv/YFjzf4rjC tiTzQ7V+YT9fzg5+/bW/pr7MLG6TzTO6YSLlP3Ypis8NbdDFMnMn2n55dfaAGoVH6ASw q2qg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=O+5+n98I; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id u13-v6si17927317pgg.263.2018.08.13.10.21.04; Mon, 13 Aug 2018 10:21:05 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=O+5+n98I; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from layers.openembedded.org (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id 8548578F59; Mon, 13 Aug 2018 17:21:00 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by mail.openembedded.org (Postfix) with ESMTP id A54DB78F4C for ; Mon, 13 Aug 2018 17:20:59 +0000 (UTC) Received: by mail-wm0-f66.google.com with SMTP id t25-v6so9581623wmi.3 for ; Mon, 13 Aug 2018 10:21:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id; bh=KYofho2ZQUbFeJJuHI7t/vCnFmL2ROvPindU0czAs9I=; b=O+5+n98IXoWtUlkvdwF7akLPKqKWnOT90lRJUDM77QHm1FFEoWu9QuqjXmpsx2MCj6 ZJvfOuLkQm96bbH93Kc9g7OTh/P8TMQ1lknrRoxKbRiW0Zr0nbCn6eVd/fq6X/NGzleW Z4wfi8ummzb3dweXWhIEbJPBIeBphg+raZ1XHH0v9T0epWI5nMMMzScbLBUiVfG0HvRm Hf8IHUgYZlASULFCeykeT39gRo3FQRRDqXmcNIiV9idzMaDYp9MLB/jgR6slPMotwGHV apF7CSpr52Ob/mzVsr+kk4d5nsi4aMIxV2lPinuhYBU5Y8lZT0iQNXaxpL6ZwidrDKqp chVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=KYofho2ZQUbFeJJuHI7t/vCnFmL2ROvPindU0czAs9I=; b=RvCFC/XS0hXJlAYzMM328bDPLMxtIMSw6DJy3Af3ZvBlNByTLO8Vw+fD0ge34RX9PO 2OM30uveiTpX2LrZ5L3M1Q4aieGe8gM9Kv7CRK1Z3ON8Y28f1GByG56K/O7c9Eo8jvlb OGyRcUmh3KRup/KwPgXks8pua9OhlFFtRPgMe4z7MlkT4nWZRk9pR++RLIZ6PjCdqiUn NJap85vW4VUma3JC4kZkbNJEeMlBHT1RuMBAoNVNlu76S/HKoRlU9e7FtN4Obb+vpEgz AKSnna/dKntSDPtLATY/DPmZJOos64VVhlhTCa5NziD29LiHm+hkHsC5ylvt9dTfLluq 3i2g== X-Gm-Message-State: AOUpUlGDVqOgFMccPnWMzntEtgCIeo1mW3G9ekzthVrbsSCVBywWAOEk nMaB5/yx/Qd30EFWTF0aQ/zs79Q3qPM= X-Received: by 2002:a1c:f306:: with SMTP id q6-v6mr8488236wmq.111.1534180859940; Mon, 13 Aug 2018 10:20:59 -0700 (PDT) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id e141-v6sm16598921wmd.32.2018.08.13.10.20.58 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Aug 2018 10:20:59 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Mon, 13 Aug 2018 18:20:52 +0100 Message-Id: <20180813172054.17767-1-ross.burton@intel.com> X-Mailer: git-send-email 2.11.0 Subject: [OE-core] [PATCH 1/3] utils/md5_file: don't iterate line-by-line X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org Opening a file in binary mode and iterating it seems like the simple solution but will still break on newlines, which for binary files isn't really useful as the size of the chunks could be huge or tiny. Instead, let's be a bit more clever: we'll be MD5ing lots of files, but we don't want to fill up memory: use mmap() to open the file and read the file in 8k blocks. Signed-off-by: Ross Burton --- bitbake/lib/bb/utils.py | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) -- 2.11.0 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/bitbake/lib/bb/utils.py b/bitbake/lib/bb/utils.py index 9903183213b..b20cdabcf01 100644 --- a/bitbake/lib/bb/utils.py +++ b/bitbake/lib/bb/utils.py @@ -524,12 +524,17 @@ def md5_file(filename): """ Return the hex string representation of the MD5 checksum of filename. """ - import hashlib - m = hashlib.md5() + import hashlib, mmap with open(filename, "rb") as f: - for line in f: - m.update(line) + m = hashlib.md5() + try: + with mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm: + for chunk in iter(lambda: mm.read(8192), b''): + m.update(chunk) + except ValueError: + # You can't mmap() an empty file so silence this exception + pass return m.hexdigest() def sha256_file(filename): From patchwork Mon Aug 13 17:20:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 144037 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp3334792ljj; Mon, 13 Aug 2018 10:21:21 -0700 (PDT) X-Google-Smtp-Source: AA+uWPypVD1Vqm4AAtHV9N1GhfI9XHncGPgH/uh7Q7a04wDDwsEM+0uec03LCiy4t/FnXlwTjEhu X-Received: by 2002:a63:530b:: with SMTP id h11-v6mr17988937pgb.139.1534180881549; Mon, 13 Aug 2018 10:21:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534180881; cv=none; d=google.com; s=arc-20160816; b=azyMmtRZDt8XShUT9tR6s9mh0kFNoCxXVhqLGJjo68JLPJdhZdKyilBLRe3kNCiC2S dgUB+oxMhLsC1/A8tuemkrZ3S1azlzzQGfRpWswJp46FWanxGkqlKFkNmxD+ElnyZRPU zIAqQj2dFNMqyd2PM2CsfS8RmRw7bcFPxZNaE0qgMcLyHuIw35uETvQx4hy+DYsWC++b 7tCSylGv49v6eeb137K1IFvHEiT4kqwXThH6cOuIm0VreDHWvXe/pWoskgLobcS1HYFY 4VyHJz1QHpulztyMDTqQEFFpxZuCsbykSOoPxQvrbWqGFVPwYqUQGmzFmv13bXMF8YBk 8a5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=Uzi849bS3Or77+ZPlcZ0UCv4c6AW6dTZTtnDpCtrZ2s=; b=nQFxutrmMKUEzwq5PArGtaqAketjMALMaFO+faMLvpSOelbhZsLIe0HmKaupimpVS/ EqEIZuudQtNOH2Zq7kYi9LzjV7ZHMuHJBfcmvK+w4lQq67h2VluYrBShMZXRh6PCkWFz aZ5KqzIj98XSK6IXgISu7DD6XGBRvyo6DUGJWgGd2jRIJdPv4kHVh8PyNJNmKxpgOhYx 4eAFavj9I47YtMOvdc+hVKanjerSBGmTTJ9mkf7o4iZTGmd2u6tcMKC47yv37HPsyqBW YTYtkgzOpwbUNg04SKUGaXFtFTqI9cZ/QPUBhCnrBKgg/3EF6mhpgtMxMW4E5uOzO6r6 9sLQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=fQjPqKux; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id g35-v6si16822538pgm.54.2018.08.13.10.21.21; Mon, 13 Aug 2018 10:21:21 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=fQjPqKux; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from layers.openembedded.org (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id A4C2678F63; Mon, 13 Aug 2018 17:21:18 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wm0-f44.google.com (mail-wm0-f44.google.com [74.125.82.44]) by mail.openembedded.org (Postfix) with ESMTP id 4992E78F5B for ; Mon, 13 Aug 2018 17:21:01 +0000 (UTC) Received: by mail-wm0-f44.google.com with SMTP id t25-v6so9581697wmi.3 for ; Mon, 13 Aug 2018 10:21:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=ty/l5H9CCdR0iznmgpFCcbV1lOMgOvhEtwvqyueAF4w=; b=fQjPqKuxl2AePGmmKagTybQBqZQ0Pi6hO5FkCiE35mdxHtIsZjARhG+Rby0Xhf9adr a3pCzpPQIAB8sDrOWEnPBipkcf3BM16NaVgkAgiQwLCDhLLrEiFf7xYtn4eeN+VhW+Qu aLVYmb2vsWAy/5zdK5XNh/GLvZ02PzHWWdQCn5TSjpzbhxZlfc3GZk50nZqOb6ZlkDFq N4VMBjuz4jomfWMJnWcGGF7PxYBathjsaHZeibXRD2k93ykfcDmGfPlulUYpgA1jTi39 7mJOMpvwwREvN0fyrNxZ3HeenDW4YQ4UyNW891mtDzREFFjaP9cqYTfOVsKZjlfecPFH RqiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ty/l5H9CCdR0iznmgpFCcbV1lOMgOvhEtwvqyueAF4w=; b=fz3fOdzTtJkk/u82kVTMfYzMF5BtFhq48NWk44g2fBAulY9PkFIpkvFpA3s1XRrN0t xnLieqSnzsvCKmRylFIYk0HdIagxoE93j3oW0ULSNoVWlBiFB3g6gC1zM8vUB0LG7klz dsHNuK8Eps+KDf4wO2Kp9POA3zCBTav47Z7KYLTRiRQVysTobqtJRG2H0sUoeIQd5bUM j98uY8ENBTFy0GrvNoAnRt34JjH9aUfxYEv9zbIuGTOsKy9cw8s+FkLCnyzIqrNfWSAc CxrVe+b/MTC726wCuVpbIv/gqJXNiC98vfCU0beURc6GK+yZBclIa69RLqHfQ49nH10D XdWg== X-Gm-Message-State: AOUpUlFYUtiVNlT9iuPN9FyqRfLGGC30cxnYCN8Fpx5XSo4jKa3eZmgK v5/nge0HdMPuaNrVgRo/PpUCSLo04Wk= X-Received: by 2002:a1c:4143:: with SMTP id o64-v6mr9272754wma.123.1534180861698; Mon, 13 Aug 2018 10:21:01 -0700 (PDT) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id e141-v6sm16598921wmd.32.2018.08.13.10.21.00 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Aug 2018 10:21:01 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Mon, 13 Aug 2018 18:20:53 +0100 Message-Id: <20180813172054.17767-2-ross.burton@intel.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180813172054.17767-1-ross.burton@intel.com> References: <20180813172054.17767-1-ross.burton@intel.com> Subject: [OE-core] [PATCH 2/3] checksum: sanity check path when recursively checksumming X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org In case something goes tragically wrong, catch a request to checksum / and refuse. Signed-off-by: Ross Burton --- bitbake/lib/bb/checksum.py | 2 ++ 1 file changed, 2 insertions(+) -- 2.11.0 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/bitbake/lib/bb/checksum.py b/bitbake/lib/bb/checksum.py index 84289208f44..4e1598fe835 100644 --- a/bitbake/lib/bb/checksum.py +++ b/bitbake/lib/bb/checksum.py @@ -97,6 +97,8 @@ class FileChecksumCache(MultiProcessCache): def checksum_dir(pth): # Handle directories recursively + if pth == "/": + bb.fatal("Refusing to checksum /") dirchecksums = [] for root, dirs, files in os.walk(pth): for name in files: From patchwork Mon Aug 13 17:20:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 144038 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp3334865ljj; Mon, 13 Aug 2018 10:21:26 -0700 (PDT) X-Google-Smtp-Source: AA+uWPyxCa4iEnwLLwhylN1i8ZuOQvcCVh8vRuIgZ14p1siqG4al/QCzqbW30+QxrwRqG+OL2hCG X-Received: by 2002:a63:4e5f:: with SMTP id o31-v6mr15468548pgl.256.1534180886326; Mon, 13 Aug 2018 10:21:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534180886; cv=none; d=google.com; s=arc-20160816; b=gf2AU96OuNsCMUI4hj3R9CdXpdwpOA/nOWo7ZGEr4qYbaz/UE2FvvF4edwZ9abgfHo /k6fflQHIFMP2MurA9kraXOfm+4l+0OfwqP8TTzopP2JjXx09iv28E+FYItkEKxf95SI BNVNQX22NA2IUzY4n3JYcdalKrVX5p3Lfw9TL1T0vjfWUFMdTpzUuxdV6Edxr6Rv4XFx QoMK77Lli43mhSBKx5EzUq3QHpEGzygOUGtW+zJqpqTMFCZL8uLR92IO0vwM39ZcBClN kGJHCJe/5BsT7jL7BZfNZr+gaxNVh94Opv0l2k2NEC+4oqL1ctuk4p+xwG9EaePKiRCw L1Fw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=MCB4hLf6YA8DGyFueMwWMAk3WUC7lvL/Aif4/hoMo0I=; b=1DZRJtkQCjsP+mxwGUTad5XdotmkO6G/CLs7kLG64NgBAthtNwTQ50JXegp1+6UKqe T0sVy+6tQmvsiEnv4KZgO1lEY0bZeT4hlM8TY/q+M3rCDxG7f7ERkI/kn+6tFWe9Ugh5 kZuSRI+cN4Z9yvMgp/cf8G7LzWj7Izh3N1+9v+PE4DJuvcc9QO23evmJMRWy84Kt9VIK xwmKleizJdGtMyf/nNSzsxEiQv/sNnJCiRierT/kdQGIUPc4VpSBGZbfGg+uVSIWxXvs b6GLTEsD6RwUoDy8SoVF57Er87KE/7ccjn59WncWrJHaZrOYGX4P80EaFSJRFt2Ox+m9 Q7VQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=dgnoGxom; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id n16-v6si18007026pgl.596.2018.08.13.10.21.26; Mon, 13 Aug 2018 10:21:26 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=dgnoGxom; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from layers.openembedded.org (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id 3A3B278F82; Mon, 13 Aug 2018 17:21:20 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wm0-f67.google.com (mail-wm0-f67.google.com [74.125.82.67]) by mail.openembedded.org (Postfix) with ESMTP id 9C0F878F5D for ; Mon, 13 Aug 2018 17:21:02 +0000 (UTC) Received: by mail-wm0-f67.google.com with SMTP id q8-v6so9618676wmq.4 for ; Mon, 13 Aug 2018 10:21:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=HtK2GT2ji5jE7OrQDMHUWwxWEoJ0RZqvH3WFklB8Ngk=; b=dgnoGxomF3MwLE1lkhtlfESs3Ye1QerqGvB+M6g4tMr9VV0D8vcxGV99JICKlNgKVw SKFkmRXQ5/e4HGMz7P0VRt8g3ajiX7eIcsNRFb0ReNBrzyB21tEL6aHyV6ON4SUXBHMC hBLD/dcVMSzNWLVjT0V+K8CYUW64bUoGiiyejPWZNUelVpyyvqDbI9iPJgzOTBmtw7sf xxuLCVWUgy8LkEAgIgaeLVBKoZyFPq3XjbAdf9oBKXNqxgXIB4uq0I50Q2WKfUqdIuPu IRvbLS69acMpllB9ajOeQW/yu3M7P8sFqNRkupFQY/6PfmYSAuBG/tU7P2OVI5SQGWNi Ss7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=HtK2GT2ji5jE7OrQDMHUWwxWEoJ0RZqvH3WFklB8Ngk=; b=EpMXkaXU0d4zr1CGKh9jV9cPtqawhjmw7pqM4bKUqDessnzmGxhVVKGxVq4LW5CE2W Mzvlaem/OrBE1qdBDvUOkg384pfH7wqJFrxoOO1U6DfIH0lkvcIPgbNmn9E6/SSkJ6Qa nxy1oGNQcJQMF2NqjYKzUDYd4NMU3VroApN2tJtUVTjSV6BDd66MLC6S1GFyxWEhIECc mUmu8eVyfwBxvWmaMpeEPYLscxzWNTCCoQ8XA6qM7GC/ut2RdrfYN+59R4Mn8tkiXwoQ CNgnRAI1+O0xzCG/jds7qGEaYHuqaS5BUWivpwj6BzkJvwPP+jeaRopnZdoNekKohD4Q IwbQ== X-Gm-Message-State: AOUpUlGqsZUdxR+Tz1VGXe/lPgFQXlS3oqCgOwZUQHFHbRhYlL1RhIBg rnt3qg0ih2NcTzSrLjAZPHmh5qC1RE0= X-Received: by 2002:a1c:cf0a:: with SMTP id f10-v6mr8340716wmg.32.1534180863109; Mon, 13 Aug 2018 10:21:03 -0700 (PDT) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id e141-v6sm16598921wmd.32.2018.08.13.10.21.01 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Aug 2018 10:21:02 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Mon, 13 Aug 2018 18:20:54 +0100 Message-Id: <20180813172054.17767-3-ross.burton@intel.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180813172054.17767-1-ross.burton@intel.com> References: <20180813172054.17767-1-ross.burton@intel.com> Subject: [OE-core] [PATCH 3/3] classes: sanity-check LIC_FILES_CHKSUM X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org We assume that LIC_FILES_CHKSUM is a file: URI but don't actually verify this, which can lead to problems if you have a URI that resolves to a path of / as Bitbake will then dutifully checksum / recursively. [ YOCTO #12883 ] Signed-off-by: Ross Burton --- meta/classes/base.bbclass | 4 ++-- meta/classes/license.bbclass | 4 +++- 2 files changed, 5 insertions(+), 3 deletions(-) -- 2.11.0 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass index 1a359a04b78..df11c8b270e 100644 --- a/meta/classes/base.bbclass +++ b/meta/classes/base.bbclass @@ -100,8 +100,8 @@ def get_lic_checksum_file_list(d): # We only care about items that are absolute paths since # any others should be covered by SRC_URI. try: - path = bb.fetch.decodeurl(url)[2] - if not path: + (method, host, path, user, pswd, parm) = bb.fetch.decodeurl(url) + if method != "file" or not path: raise bb.fetch.MalformedUrl(url) if path[0] == '/': diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass index 2ac15a1bc89..4cf7f074fc2 100644 --- a/meta/classes/license.bbclass +++ b/meta/classes/license.bbclass @@ -205,7 +205,9 @@ def find_license_files(d): for url in lic_files.split(): try: - (type, host, path, user, pswd, parm) = bb.fetch.decodeurl(url) + (method, host, path, user, pswd, parm) = bb.fetch.decodeurl(url) + if method != "file" or not path: + raise bb.fetch.MalformedUrl() except bb.fetch.MalformedUrl: bb.fatal("%s: LIC_FILES_CHKSUM contains an invalid URL: %s" % (d.getVar('PF'), url)) # We want the license filename and path