bs4

domonic.bs4 gives you a Beautiful Soup 4 style API while keeping a real domonic DOM underneath.

The important bit is that returned objects are ordinary domonic nodes. They are not wrapper Tag objects and there is no second parsed tree.

Importing domonic.bs4 installs the compatibility methods as an opt-in patch onto domonic’s Node, Element, Document, DocumentFragment, Text and Comment classes. A normal import domonic does not install these methods.

Why BeautifulSlop?

Use BeautifulSlop when you want Beautiful Soup style scraping code but you also want actual domonic nodes underneath:

• familiar find(), find_all(), select(), get_text() and mutation methods

• normal querySelectorAll(), parentNode, appendChild(), setAttribute() and XPath access

• fast query-heavy workflows over parsed HTML

• no wrapper Tag objects and no duplicate tree

Parse

from domonic.bs4 import BeautifulSlop

markup = """
<main id="site">
  <article class="post featured">
    <h1>Hello</h1>
    <p>Read <a href="/docs" class="external">the docs</a>.</p>
  </article>
</main>
"""

soup = BeautifulSlop(markup, "html.parser")

BeautifulSlop accepts domonic parser names. The default is "auto", which picks the fastest installed backend (see Parser Performance); pass a name to pin one:

BeautifulSlop(markup)                 # "auto" -- fastest available
BeautifulSlop(markup, "html.parser")
BeautifulSlop(markup, "html5lib")
BeautifulSlop(markup, "lxml_html")
BeautifulSlop(markup, "markupever")
BeautifulSlop(markup, "selectolax")
BeautifulSlop(markup, "turbohtml")
BeautifulSlop(markup, "html5_parser")
BeautifulSlop(markup, "justhtml")
BeautifulSlop(markup, "expat")

Text extraction (get_text(), strings, stripped_strings) on a whole page reads the parser’s own record of the text nodes rather than walking the tree, until the tree changes.

It also accepts familiar Beautiful Soup names where they map cleanly onto domonic parser backends:

BeautifulSlop(markup, "html.parser")  # Python stdlib parser
BeautifulSlop(markup, "lxml")         # maps to lxml_html

Install optional parser backends as needed:

python -m pip install selectolax
python -m pip install turbohtml
python -m pip install lxml
python -m pip install markupever

Scrape Links

from domonic.bs4 import BeautifulSlop

html = """
<article>
  <a href="/docs" class="external">Docs</a>
  <a href="/api">API</a>
</article>
"""

soup = BeautifulSlop(html, "html.parser")

for link in soup.find_all("a", href=True):
    print(link.text, link["href"])

Find Tags

first_link = soup.find("a")
all_links = soup.find_all("a")

article = soup.find("article", id="main")
external = soup.find_all("a", class_="external")
text_inputs = soup.find_all("input", {"type": "text"})

for link in all_links:
    print(link.get("href"))

Limit results:

first_two_links = soup.find_all("a", limit=2)

Search only direct children:

article.find("a", recursive=False)
article.find_child("h1")
article.find_children()

Filter Values

String filters:

soup.find("a", href="/docs")
soup.find("article", class_="post featured")

List filters:

soup.find_all(["article", "aside"])
soup.find_all("p", {"class": ["lede", "summary"]})

Regular expression filters:

import re

soup.find(re.compile("^art"))
soup.find("a", href=re.compile(r"docs$"))
soup.find("p", class_=re.compile("featured"))

Callable filters:

soup.find(lambda tag: tag.name == "article")
soup.find("a", href=lambda value: value and value.startswith("/"))
soup.find("p", class_=lambda value: value == "featured")

Presence and absence filters:

soup.find_all("a", href=True)
soup.find_all("img", alt=None)

Text Search

Use string= for Beautiful Soup style text matching:

title_text = soup.find(string="Hello")
docs_text = soup.find(string=re.compile("docs"))
heading = soup.find("h1", string="Hello")

The older text= alias is also accepted:

heading = soup.find("h1", text="Hello")

domonic may return Text nodes for parsed text. They render as text and still remain part of the real domonic DOM:

print(str(title_text))

CSS Selectors

select and select_one delegate to domonic’s selector engine:

soup.select("article a.external")
soup.select_one("article > p")
soup.select('a[href^="/docs"]')
soup.select("ul > li:nth-child(2)")
soup.select("p:first-child, p:last-child")

Repeated selector workflows are a good fit for Slop: the objects remain domonic nodes, common selector paths avoid a second CSS engine, and queries over an unchanged tree are served from lazily-built tag / class / attribute indexes on the root rather than a fresh walk. find_all calls that reduce to a tag (or list of tags) plus attribute-presence filters – find_all(["a", "span"]), find_all("a", href=True), find_all(attrs={"data-role": True}) – come straight from those indexes. The indexes rebuild automatically after any mutation, whether through a Slop method or the underlying DOM API. See Parser Performance.

Navigation

link = soup.find("a")

link.parent
list(link.parents)

link.next_sibling
list(link.next_siblings)

link.previous_sibling
list(link.previous_siblings)

link.next_element
list(link.next_elements)

link.previous_element
list(link.previous_elements)

Search relative to a node:

link.find_parent("article")
link.find_parents()
link.find_next("p")
link.find_all_next("a")
link.find_previous("h1")
link.find_all_previous()
link.find_next_sibling("aside")
link.find_previous_sibling("p")

Attributes

link = soup.find("a")

link.attrs
link.get("href")
link.get("missing", "fallback")
link.has_attr("href")

link["href"] = "/new-url"
print(link["href"])
del link["href"]

The old has_key alias is available for compatibility:

link.has_key("href")

Text Extraction

article = soup.find("article")

article.text
article.get_text()
article.get_text(" | ", strip=True)

list(article.strings)
list(article.stripped_strings)

Mutation

Use familiar Beautiful Soup methods, backed by domonic’s DOM operations:

article = soup.find("article")

badge = soup.new_tag("span", class_="badge")
badge.append("new")

article.find("h1").insert_after(badge)
article.append(soup.new_tag("footer"))

Remove nodes:

aside = soup.find("aside")
removed = aside.extract()

soup.find("script").decompose()
soup.find("article").clear()

Replace or wrap nodes:

old = soup.find("b")
strong = soup.new_tag("strong")
strong.append(old.text)
old.replace_with(strong)

paragraph = soup.find("p")
wrapper = soup.new_tag("section", class_="copy")
paragraph.wrap(wrapper)
wrapper.unwrap()

Smooth adjacent text nodes:

p = soup.find("p")
p.append(soup.new_string(" more"))
p.smooth()

Create Nodes

tag = soup.new_tag("a", href="/hello", class_="external")
tag.append("hello")

text = soup.new_string("plain text")

These return domonic Element and Text objects.

Still Domonic

Because BeautifulSlop returns normal domonic nodes, the DOM API remains available:

article = soup.find("article")

soup.querySelectorAll("article > a")
article.parentNode
article.setAttribute("data-ready", "yes")
article.getAttribute("data-ready")

Render

print(soup)
print(soup.prettify())

Related Examples and Guides

• Scrape HTML

• Parser Performance

• examples/beautifulslop.py

• scripts/benchmark_bs4.py

Compatibility Notes

This layer targets the common Beautiful Soup workflows: search, navigation, attributes, text extraction, creation, mutation and rendering. It does not try to clone Beautiful Soup internals such as builders, formatter objects, encoding detection, result set subclasses or NavigableString.

domonic.bs4

Beautiful Soup 4 style convenience methods for domonic nodes.

Importing this module opt-in patches a small, familiar BS4 API onto domonic’s real DOM classes. Returned objects remain normal domonic nodes, not wrappers.

class domonic.bs4.BeautifulSlop(markup='', features=None, *args, **kwargs)

Bases: object

Parse markup and return a domonic tree with BS4-style helpers.

Return type:

Node

domonic.bs4.install()

Install the BS4 compatibility API onto domonic node classes.