From patchwork Tue May 16 13:20:48 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 99871 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp2017227qge; Tue, 16 May 2017 06:21:16 -0700 (PDT) X-Received: by 10.84.217.201 with SMTP id d9mr15576476plj.164.1494940876101; Tue, 16 May 2017 06:21:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1494940876; cv=none; d=google.com; s=arc-20160816; b=vA9z3GFc01MggwbgOZ6qbsF6rN9O5SaKitr+mGSf3RB+bP8gmC51MHdZKB3RqvO21M xTJWcNPpr1Zx6EDH4Xf0TWi6jSOe78O/uV6YMDS23e/Rn97hj8uNUgllUbKPioZ6xw+E 0zSpOoTKtPl7/ra4d71wMzLh3lAUOLjN8Y9H679jj0S3bjYzNNY5RBTfm6023bQN0pFZ d7t6hrXvxfdLSH252goCw6D3OyWKlRD9E5lDRpLVHYHVZTazGVGipfCjRZu3rqELfm0M SBijxXBYX5tkl2DAw76gB3bEW3vC3aQIqXyxBcC/ZiWQpKWktwz9LzU5LXs8pxFQqeFZ VzJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=urvaAv4USQqBtfTvwxLqWZ6Zh4JKfpkQ2QL3jC0kYAQ=; b=HBYznFuhvqRbx02W6ulAdyR1GuAFFiYOmETXaE/TVmPt+iDcU4L8QtfJy2NXxKjtJt yXt0vW/ozFAjPxZnw5h1P3v7EeAODtbq/RH7tNbUwYj0GaUJI/KyDMVUiwTopiaXzNhB 6fSdRhcx0KGs+iYaZGzW7yQerMrWhLhE74namTEAgaFQ+YMV30kTqS0pF679Ru0nFcOb ROWLpxJ7mVjJTTKl9PdTdenOutuQIBIwSWh6XxfvoHgW8cNFtJDRhWYAg91Y1vZDd2ya +5l74/qoOTkjfvq8tXs6q109XiobN4dEJAEMy4CFGGEIXMAdNIwsMNA41oBTq9mkbRZI DfKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id 79si13110523pfx.98.2017.05.16.06.21.15; Tue, 16 May 2017 06:21:16 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org Received: from review.yoctoproject.org (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id E9EAF77E58; Tue, 16 May 2017 13:20:54 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wm0-f54.google.com (mail-wm0-f54.google.com [74.125.82.54]) by mail.openembedded.org (Postfix) with ESMTP id 034E877E56 for ; Tue, 16 May 2017 13:20:52 +0000 (UTC) Received: by mail-wm0-f54.google.com with SMTP id b84so133394498wmh.0 for ; Tue, 16 May 2017 06:20:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id; bh=i/laglanoAtk/8uTzzSl+TlKmJiq4+X+Ovu7Gw3qvuo=; b=1DpZTzW5kvAayeeU3qGuYmJRBWDexvlJ8gajw4Oa1H9OHx8Vvj+sqG1ncdBjvXpJFH XHQfgur74AQbxtCPwnt2K+yhAPpO8bFEEYcbhA6ZDxTBeAfAY9fGmP1D3suMgCgysMMh Sk4KZ7KnKGD10IO2iCbLyU25EzCTo8Uelk4KSjFAVbygab/DhTtsnud9FA7telOZnh+K 7mqE0EUYoma/1bLg77GeFZjZGGBld8pYy/Hb1+EggX5V0OJs7WDRO8mOladjIBOf9Ima 0m3HfDztiK/kxzfgJs7UjJmWZ8o6wDY5hCwYTlsacFHND5gFEUIQA7F5kZhU2ZXX5SAM IYRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=i/laglanoAtk/8uTzzSl+TlKmJiq4+X+Ovu7Gw3qvuo=; b=Aa0pIMUH5k7utm+CXNdM9EbY/3A5l9rpoqdBaLM5XQKRWDmLGewMfgIRfKle6DM8RH B2zhee/+I0n233/vKLpbfs+lagy2QNL6pw0Z0hGC9ig+v3b87vn4yUQ4tjz/eRyMYG+d r6TpwIb3PdbNUwoekq5UM90Wewgw0/CFuof6zGx8+rW77IHr+92MLEI+gqWDX0ZgtTXG OfON6egq8VduvSbnTLwMe/1xmsd/LPyNGIjg0kkQmL59gXzrgChWgCUKsDQFN9hXR83o B6m0zQq/GF1zERfmcSS46OF2G3OiH7Q48ymPYr4I6BOQqTGOH1KNu5dief9VEwhoMKRb dXsA== X-Gm-Message-State: AODbwcDsVCb08K5uZJ2Jbc3al9HTmBgVDQod+z2hC3bwjNsSP5EF4FmV Fs+jhCqMpf00cZoskpE= X-Received: by 10.28.16.212 with SMTP id 203mr6490635wmq.26.1494940852927; Tue, 16 May 2017 06:20:52 -0700 (PDT) Received: from flashheart.burtonini.com (home.burtonini.com. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id h70sm6579646wma.14.2017.05.16.06.20.51 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 16 May 2017 06:20:52 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Tue, 16 May 2017 14:20:48 +0100 Message-Id: <1494940848-25745-1-git-send-email-ross.burton@intel.com> X-Mailer: git-send-email 2.8.1 Subject: [OE-core] [PATCH] scripts: add pkgdataui X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org pkgdataui is a Python 3/GObject Introspection/GTK+ 3 tool to browse the pkgdata database at your leisure. By being graphical it is easier to explore and can follow links between packages. This is very much a work in progress, so be gentle and patches are welcome. Signed-off-by: Ross Burton --- scripts/pkgdataui.glade | 335 ++++++++++++++++++++++++++++++++++++++++++++++++ scripts/pkgdataui.py | 241 ++++++++++++++++++++++++++++++++++ 2 files changed, 576 insertions(+) create mode 100644 scripts/pkgdataui.glade create mode 100755 scripts/pkgdataui.py -- 2.8.1 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/scripts/pkgdataui.glade b/scripts/pkgdataui.glade new file mode 100644 index 0000000..04e987b --- /dev/null +++ b/scripts/pkgdataui.glade @@ -0,0 +1,335 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + False + Package Data Browser + accessories-dictionary + True + + + True + False + 4 + 4 + 4 + 4 + vertical + 4 + + + False + pkgdata_store + 1 + + + + 0 + + + + + False + True + 0 + + + + + True + True + 200 + True + + + True + True + in + 100 + + + True + True + recipe_store + 0 + True + False + + + + + + fixed + Recipe + + + + 0 + + + + + + + + + False + True + + + + + True + True + 200 + True + + + True + True + in + 100 + + + True + True + package_store + 0 + False + + + + + + True + autosize + Package + 0 + + + + 0 + + + + + + + True + autosize + Size + 1 + + + + + + + + + + False + True + + + + + True + False + 4 + vertical + 4 + + + True + False + 0 + label + + + False + True + 0 + + + + + True + False + 0 + depends_label + True + False + + + False + True + 1 + + + + + True + False + 0 + recs_label + True + False + + + False + True + 2 + + + + + True + False + 0 + suggests_label + True + False + + + False + True + 3 + + + + + True + False + 0 + provides_label + True + False + + + False + True + 4 + + + + + True + False + 0 + files_label + end + + + False + True + 5 + + + + + True + True + in + + + True + True + file_store + True + 0 + False + + + + + + Name + True + 0 + + + rgba(0,0,0,0) + + + 0 + + + + + + + Size + True + 1 + + + + 1 + + + + + + + + + True + True + 6 + + + + + True + True + + + + + True + True + + + + + True + True + 1 + + + + + + diff --git a/scripts/pkgdataui.py b/scripts/pkgdataui.py new file mode 100755 index 0000000..f6b23fc --- /dev/null +++ b/scripts/pkgdataui.py @@ -0,0 +1,241 @@ +#! /usr/bin/env python3 + +import os, sys, enum, ast + +scripts_path = os.path.dirname(os.path.realpath(__file__)) +lib_path = scripts_path + '/lib' +sys.path = sys.path + [lib_path] + +import scriptpath +bitbakepath = scriptpath.add_bitbake_lib_path() +if not bitbakepath: + print("Unable to find bitbake by searching parent directory of this script or PATH") + sys.exit(1) +import bb + +import gi +gi.require_version('Gtk', '3.0') +from gi.repository import Gtk, Gdk, GObject + +RecipeColumns = enum.IntEnum("RecipeColumns", {"Recipe": 0}) +PackageColumns = enum.IntEnum("PackageColumns", {"Package": 0, "Size": 1}) +FileColumns = enum.IntEnum("FileColumns", {"Filename": 0, "Size": 1}) + +import time +def timeit(f): + def timed(*args, **kw): + ts = time.time() + print ("func:%r calling" % f.__name__) + result = f(*args, **kw) + te = time.time() + print ('func:%r args:[%r, %r] took: %2.4f sec' % \ + (f.__name__, args, kw, te-ts)) + return result + return timed + +def human_size(nbytes): + import math + suffixes = ['B', 'KB', 'MB', 'GB', 'TB', 'PB'] + human = nbytes + rank = 0 + if nbytes != 0: + rank = int((math.log10(nbytes)) / 3) + rank = min(rank, len(suffixes) - 1) + human = nbytes / (1000.0 ** rank) + f = ('%.2f' % human).rstrip('0').rstrip('.') + return '%s %s' % (f, suffixes[rank]) + +def load(filename, suffix=None): + from configparser import ConfigParser + from itertools import chain + + parser = ConfigParser() + if suffix: + parser.optionxform = lambda option: option.replace("_" + suffix, "") + with open(filename) as lines: + lines = chain(("[fake]",), lines) + parser.read_file(lines) + + # TODO extract the data and put it into a real dict so we can transform some + # values to ints? + return parser["fake"] + +def find_pkgdata(): + import subprocess + output = subprocess.check_output(("bitbake", "-e"), universal_newlines=True) + for line in output.splitlines(): + if line.startswith("PKGDATA_DIR="): + return line.split("=", 1)[1].strip("\'\"") + # TODO exception or something + return None + +def packages_in_recipe(pkgdata, recipe): + """ + Load the recipe pkgdata to determine the list of runtime packages. + """ + data = load(os.path.join(pkgdata, recipe)) + packages = data["PACKAGES"].split() + return packages + +def load_runtime_package(pkgdata, package): + return load(os.path.join(pkgdata, "runtime", package), suffix=package) + +def recipe_from_package(pkgdata, package): + data = load(os.path.join(pkgdata, "runtime", package), suffix=package) + return data["PN"] + +def summary(data): + s = "" + s += "{0[PKG]} {0[PKGV]}-{0[PKGR]}\n{0[LICENSE]}\n{0[SUMMARY]}\n".format(data) + + return s + + +class PkgUi(): + def __init__(self, pkgdata): + self.pkgdata = pkgdata + self.current_recipe = None + self.recipe_iters = {} + self.package_iters = {} + + builder = Gtk.Builder() + builder.add_from_file(os.path.join(os.path.dirname(__file__), "pkgdataui.glade")) + + self.window = builder.get_object("window") + self.window.connect("delete-event", Gtk.main_quit) + + self.recipe_store = builder.get_object("recipe_store") + self.recipe_view = builder.get_object("recipe_view") + self.package_store = builder.get_object("package_store") + self.package_view = builder.get_object("package_view") + + # Somehow resizable does not get set via builder xml + package_name_column = builder.get_object("package_name_column") + package_name_column.set_resizable(True) + file_name_column = builder.get_object("file_name_column") + file_name_column.set_resizable(True) + + self.recipe_view.get_selection().connect("changed", self.on_recipe_changed) + self.package_view.get_selection().connect("changed", self.on_package_changed) + + self.package_store.set_sort_column_id(PackageColumns.Package, Gtk.SortType.ASCENDING) + builder.get_object("package_size_column").set_cell_data_func(builder.get_object("package_size_cell"), lambda column, cell, model, iter, data: cell.set_property("text", human_size(model[iter][PackageColumns.Size]))) + + self.label = builder.get_object("label1") + self.depends_label = builder.get_object("depends_label") + self.recommends_label = builder.get_object("recommends_label") + self.suggests_label = builder.get_object("suggests_label") + self.provides_label = builder.get_object("provides_label") + + self.depends_label.connect("activate-link", self.on_link_activate) + self.recommends_label.connect("activate-link", self.on_link_activate) + self.suggests_label.connect("activate-link", self.on_link_activate) + + self.file_store = builder.get_object("file_store") + self.file_store.set_sort_column_id(FileColumns.Filename, Gtk.SortType.ASCENDING) + self.files_view = builder.get_object("files_scrollview") + self.files_label = builder.get_object("files_label") + + self.load_recipes() + + self.recipe_view.set_cursor(Gtk.TreePath.new_first()) + + self.window.show() + + def on_link_activate(self, label, url_string): + from urllib.parse import urlparse + url = urlparse(url_string) + if url.scheme == "package": + package = url.path + recipe = recipe_from_package(self.pkgdata, package) + + it = self.recipe_iters[recipe] + path = self.recipe_store.get_path(it) + self.recipe_view.set_cursor(path) + self.recipe_view.scroll_to_cell(path) + + self.on_recipe_changed(self.recipe_view.get_selection()) + + it = self.package_iters[package] + path = self.package_store.get_path(it) + self.package_view.set_cursor(path) + self.package_view.scroll_to_cell(path) + + return True + else: + return False + + def on_recipe_changed(self, selection): + self.package_store.clear() + self.package_iters = {} + + (model, it) = selection.get_selected() + if not it: + return + + recipe = model[it][RecipeColumns.Recipe] + for package in packages_in_recipe(self.pkgdata, recipe): + # TODO also show PKG after debian-renaming? + data = load_runtime_package(self.pkgdata, package) + # TODO stash data to avoid reading in on_package_changed + self.package_iters[package] = self.package_store.append([package, int(data["PKGSIZE"])]) + + def on_package_changed(self, selection): + self.label.set_text("") + self.file_store.clear() + self.depends_label.hide() + self.recommends_label.hide() + self.suggests_label.hide() + + (model, it) = selection.get_selected() + if it is None: + return + + package = model[it][PackageColumns.Package] + data = load_runtime_package(self.pkgdata, package) + + self.label.set_text(summary(data)) + + files = ast.literal_eval(data["FILES_INFO"]) + if files: + self.files_label.set_text("{0} files take {1}.".format(len(files), human_size(int(data["PKGSIZE"])))) + self.files_view.show() + for filename, size in files.items(): + self.file_store.append([filename, size]) + else: + self.files_view.hide() + self.files_label.set_text("This package has no files.") + + def update_deps(field, prefix, label, clickable=True): + if field in data: + l = [] + for name, version in bb.utils.explode_dep_versions2(data[field]).items(): + if clickable: + l.append("{0} {1}".format(name, " ".join(version))) + else: + l.append("{0} {1}".format(name, " ".join(version))) + label.set_markup(prefix + ", ".join(l)) + label.show() + else: + label.hide() + update_deps("RDEPENDS", "Depends: ", self.depends_label) + update_deps("RRECOMMENDS", "Recommends: ", self.recommends_label) + update_deps("RSUGGESTS", "Suggests: ", self.suggests_label) + update_deps("RPROVIDES", "Provides: ", self.provides_label, clickable=False) + + def load_recipes(self): + for recipe in sorted(os.listdir(pkgdata)): + if os.path.isfile(os.path.join(pkgdata, recipe)): + self.recipe_iters[recipe] = self.recipe_store.append([recipe]) + +if __name__ == "__main__": + import argparse + + parser = argparse.ArgumentParser(description='pkgdata browser') + parser.add_argument('-p', '--pkgdata', help="Optional location of pkgdata") + + args = parser.parse_args() + pkgdata = args.pkgdata if args.pkgdata else find_pkgdata() + # TODO assert pkgdata is a directory + window = PkgUi(pkgdata) + Gtk.main()