Version 0.9.32 - 2025-08-12

The last month has featured our team banging our collective heads against the wall as we work on tightening up the outstanding skipped tests. While there are a couple of stray parsing bugs here and there, most of it has to do with ensuring that the parsing of Markdown tokens are complete... including whitespaces. This is not an issue for generating HTML, but we feel it is a big issue for people writing rules against those tokens. As such, we are trying to ensure we capture as many scenarios as possible, and making sure all tests pass. Its often a tough job, but we feel it is worth it!

On that note, please note that if you are using pragmas to temporarily turn off rules in your documents, there is a small change. In fixing Issue 1447, we noticed there was an issue with how the pragmas ended. To fix this properly, as of release 0.9.32, pragmas starting with a <!-- must end with a --> and pragmas starting with a <!--- must end with a --->. This should be a one time change, and we apologize in advance for any issues it causes.

A big thanks to our users who are reporting issues with the project! While we would like to believe we can think of every test case, we know we cannot. Therefore, we continue to need our users to help us out. If you are scanning any Markdown documents and the results seem off, please file an issue. If you are starting to use our fix mode on your Markdown documents and there are issues, please file an issue. We appreciate any help that we can get to improve the project for everyone!

Added

• Issue 1454
  • added support for --enable-extensions to the command line and API
• Issue 1455
  • added API functions for --no-json5 and --continue-on-error to achieve parity with command line

Fixed

• Issue 1387
  • fixed weird indent issue with Md005
• Issue 1400
  • addressed issues with Md041 and how it handles HTML Block elements at the start of a document
  • added extra examples to documentation to expand on some interesting areas with the Md041 rule and "invisible" tags
• Issue 1441
  Issue 1443
  • an invalid LRD, started right after a container block was started did not unwind properly
• Issue 1446
  • multiple issues with LRDs and whitespace/tabs
• Issue 1447
  • was not handling ---> properly at the end of pragmas
• Issue 1464
  • adding better support for tabs with LRDs.

Life caught up with our development team this last two months, but we finally finished the work we intended to do for this release. After spending some time to upgrade application_properties to 0.9.0, this release of PyMarkdown adopts that new version, including the simplified setup that was added to application_properties. In addition, PyMarkdown is not using the JSON5 support from application_properties and defaults to it being on. While completing that work, we also went through the configuration documentation, delegating any duplicated sections with the application_properties documentation to application_properties .

During this time, we did some internal housekeeping. We maintain a small group of packages, with common scripting and support to help our team use best practices across all packages. That had not been maintained lately, so we chose to spend a couple of weeks working on getting the packages back to where we wanted them. No effect for how you use this package, but it helps us keep things current.

A big thanks to our users who are reporting issues with the project! While we would like to believe we can think of every test case, we know we cannot. Therefore, we continue to need our users to help us out. If you are scanning any Markdown documents and the results seem off, please file an issue. If you are starting to use our fix mode on your Markdown documents and there are issues, please file an issue. We appreciate any help that we can get to improve the project for everyone!

Added

• Issue 1378
  • upgrade to latest version of application_properties, v0.9.0
  • enabled ability to use JSON5 (default) or normal JSON via command line
    switch
  • made small modifications to plugins as our team is unaware of any
    custom plugin users
    • change is to use a local MyApplicationPropertiesFacade which
      provides better abstraction
  • removed redundant configuration documentation in favor of pointing
    links at updated application_properties documentation
• Issue 1427
  • Adhering to project templates across various projects.

Fixed

• None

Changed

• None

Version 0.9.30 - Date: 2025-04-19

Apologies for a late release, but we wanted to get some outstanding issues resolved before the release. Notable things to mention are:

• Blanket disabling and then selective enabling of rules
• New pragma disable and enabled commands
• Exclusion of specific files and glob file paths from scanning

And while it may not be visible, a lot of work has been completed on getting proper Table token support in the application. As the rules engine is largely token based, any desired rules that include tables must rely on well-tested support for the new table token. Our development team is hoping to make more progress on this in the next couple of months, and will announce it loudly when it is completed!

A big thanks to our users who are reporting issues with the project! While we would like to believe we can think of every test case, we know we cannot. Therefore, we continue to need our users to help us out. If you are scanning any Markdown documents and the results seem off, please file an issue. If you are starting to use our fix mode on your Markdown documents and there are issues, please file an issue. We appreciate any help that we can get to improve the project for everyone!

Added

• Issue 1396
  • Support for specific rule enablement
  • Disabling of all rules and then selective enabling
• Issue 1401
  • Ability to exclude files from the command line
• Issue 1403
  • Added disable and enable for pragmas

Fixed

• Issue 1379
  • Fixed rehydration issues for test_extra_052l* and test_extra_052m*
  • Precursor for addressing the fix mode Md031 tests
• Issue 1380
  • Fixing the 2 Md031 tests now unblocked from Issue 1379
• Issue 1387
  • Fixed Md005 not firing properly within alternating ordered and unordered list
• Issue 1401
  • Fixed a small issue with the ** glob characters not working properly

Changed

• None

The work continues! Due to our development team passing colds back and forth between members, February was a slow month in terms of progress. We plan to start adding the foundations for new leaf items in March and April, in addition to completing the current round of issues found with rule Md031's fix mode.

A big thanks to our users who are reporting issues with the project! While we would like to believe we can think of every test case, we know we cannot. Therefore, we continue to need our users to help us out. If you are scanning any Markdown documents and the results seem off, please file an issue. If you are starting to use our fix mode on your Markdown documents and there are issues, please file an issue. We appreciate any help that we can get to improve the project for everyone!

If everyone reading this can take a moment and think of our mascot, "Bruce" and any pets of their own that have moved on, it would be appreciated. He passed away this morning suddenly. Our team can honestly say that Bruce was the best rubber duck that we have ever had, and we will miss him very much.

Added

• None

Fixed

• Issue 1352
  • Fixed parsing and rehydration issues, as precursor to tackling Md031 fix issue
• Issue 1356
  • Fixed rehydration error with test test_extra_052r0.
• Issue 1358
  • Fixed rehydration error with test test_extra_053d1.
• Issue 1364
  • Fixed parsing error and rehydration error related to multiple drop
    container scenarios involving the remaining containers being lists.
• Issue 1357
  • Fixed issue with rule Md007 and nested lists
• Issue 1371
  • Fixed Md031 fix issues related to issue 1364 fixes.

Changed

• None

The work continues! Working through the categorized list takes time, but it is well worth it. We are also working on improving our bash and github actions infrastructure, making it easier to release the project each month.

A big thanks to our users who are reporting issues with the project! While we would like to believe we can think of every test case, we know we cannot. Therefore, we continue to need our users to help us out. If you are scanning any Markdown documents and the results seem off, please file an issue. If you are starting to use our fix mode on your Markdown documents and there are issues, please file an issue. We appreciate any help that we can get to improve the project for everyone!

Added

• Issue 1289
  • added documentation under Pre-Commit for how to write the configuration file for Pre-Commit if an alternate extension is needed for the files being scanned
• Issue 1318
  • ported perf_*.cmd scripts to perf_.sh scripts
• Issue 1320
  • added ShFmt .sh formatter to Pre-Commit configuration
• Issue 1322
  • added Shellcheck .sh scanner to Pre-Commit configuration
• Issue 1327
  • added simple mechanism to log and view performance measurements

Fixed

• Issue 1302
  • reported issue where C\# at the end of a header was triggering rule Md020 for no spaces between end mark of an Atx Heading
• Issue 1326
  • fix mode for MD012 not properly handling double lines in lists after new list indicators
• Issue 1288
  • rule Md037 and documentation updated to properly ensure that at least one of the two ends has spaces on both sides of the emphasis sequence
• Issue 1334
  • fixed SNAFU1 for MD031 fix mode, assert
• Issue 1344
  • fixed SNAFU6 for MD031 fix mode, assert
• Issue 1346
  • fixed SNAFU8 for MD031 fix mode, assert

Changed

• Issue 1277
  • Rule Md044 changed to add a code_spans exclusion as a compliment to the code_blocks exclusion.

Version 0.9.27 - Date: 2025-01-09

This might sound like we are repeating ourselves, but things march on. The tests for the fix mode for Md031 are still revealing secrets, and we are fixing them as fast as we can. We took some time in December to make sure we have a categorized list of what still needs to be done and have prioritized that work. Our research required for new leaf elements is almost complete. We took a bit of a detour to ensure our Bash script support and performance measuring support is in place, something we feel strongly about. Our current plans include efforts to tackle both of these items in January, along with other reported issues. 

And thanks to the users who are reporting issues with the project! While we would like to believe we can think of every test case, we know we cannot. Therefore, we continue to need our users to help us out. If you are scanning any Markdown documents and the results seem off, please file an issue. If you are starting to use our fix mode on your Markdown documents and there are issues, please file an issue. We appreciate any help that we can get to improve the project for everyone!

Added

• Issue 1299
  • adding scripts to capture performance numbers to enable tuning
• Issue 35
  • added bash scripts to compliment most of the Windows .cmd scripts, including the clean.cmd being complimented by clean.sh

Fixed

• Issue 1297
  • Fixed some Md031 fix tests, organized others.

Changed

• Issue 1290
  • Upgraded tests to support Python 3.9 to 3.13

Version 0.9.26 - Date: 2024-12-09

Progress continues on locating issues and addressing them, including four issues reported by users. As of this past weekend, (to the best of our knowledge) we have eliminated all fatal issues with Rule Md031 and its fix mode. While there are only three weeks left until the new year, we hope to make significant progress on Rule Md031's fix mode producing incorrect markdown. At the same time, we are starting to do research work into determining the best patterns for introducing new leaf elements in 2025. Our main goal is to provide thorough coverage without sacrificing proper testing. To that end, we will try and figure out and document the best approaches so we can use them for new leaf elements in 2025.

In addition to this work, make solid progress on addressing user issues as reported. And this might seem repetitive, but we continue to need our users to help us out. If you are scanning any Markdown documents and the results seem off, please file an issue. If you are starting to use our fix mode on your Markdown documents and there are issues, please file an issue. We appreciate any help that we can get to improve the project for everyone!

Added

• Issue 810
  • Added fix mode for Rule Md012
• Issue 1280
  • Added testing capability to save all single Markdown documents in a specified directory, then scanning them one at a time with PyMarkdown and each of the extensions enabled.

Fixed

• Issue 1259
  • Fixed asserts and bad parsing from cases where containers are added and then a "raw" blank line removes all containers.
• Issue 1263
  • Fixed issue where a new unordered list between two block quotes was not being recognized properly.
• Issue 1270
  • Fixed issue with Md027 not reporting line numbers properly within anything except the first paragraph.
• Issue 1272
  • Parsing of the FCB in certain cases was off, as was the text token containing the code block's text. Resulted in the columns being reported being indented less than expected.
• Issue 1274
  • Fixed remaining assert issues, leaving fixes that produce valid Markdown, but not the intended Markdown.
• Issue 1267
  • Fixed reported issue with task lists creating an error in Md018.
• Issue 1268
  • Fixed issue with Md022 and pragmas, similar to Issue 1208.

Changed

• Issue 1258
  • Moved twine package into dev section

Version 0.9.25 - Date: 2024-11-07

While it seems like we have been working on the fixing for Rule Md031 forever, that time is starting to come to an end. We have a solid list of what is left to test, and we are confident that we will finish it before the new year. (Hope we did not just jinx ourselves!) As with the last couple of releases, we are testing variations of containers, container starts, and container ends, all to ensure we have confidence that our test scenarios are thorough. At this point, we are very confident with any nesting of up to three containers, will our confidence for nesting scenarios of up to four containers at a high level as well. Following close behind that is our fix mode for Rule Md031 which is the stressor for the nested containers. We are not always happy that we started working on the fix mode for Rule Md031, but we are happy that it uncovered some issues in our parser that we could quickly fix.

But we continue to need our users to help us out. If you are scanning any Markdown documents and the results seem off, please file an issue. If you are starting to use our fix mode on your Markdown documents and there are issues, please file an issue. We appreciate any help that we can get to improve the project for everyone!

Added

• Issue 1233
• Issue 1234
• Issue 1235
  • Adding more comprehensive "drop X" tests where multiple levels of containers are involved, and then dropping one or more of those containers in a single line.

Fixed

• Issue 1208
  • Fixed issue with blank lines separated with pragmas not being understood properly.
• Issue 1233
• Issue 1234
• Issue 1235
  • Adding new "drop_x" tests and resolve any scan issues with them.
• Issue 1243
• Issue 1245
  • Handling leading spaces in __fix_spacing function now that data is present.
• Issue 1247
  • In 3+ drop cases with only lists and no block quotes, indent is not calculated properly on rehydrate. This in turn causes the fixed text to be wrong.
• Issue 1250
  • Batch of fixes for cases when Md031 is trying to properly space a fenced code block after dropping 2 containers.

Changed

• Issue 1231)
  • Moved triple nested container tests into their own test_nested_three_* files for better readability.

Version 0.9.24 - Date: 2024-10-06

This release continued our focus on enabling fixing for Rule Md031 and uncovering any issues with the more deeply nested container cases. This has meant introducing a new helper class to assist in the tracking of a given line to the container tokens used to provide container-based indenting for that line. This is very important for Rule Md031, and has already proveded to be useful in a partial rewrite of some of the logic for Md027.

While we find the odd parsing error, those issues are now rare to find in container nesting of three container or less, especially compared to finding issues with our new fix logic. Still, we continue to try different combinations of containers elements and leaf elements, verifying that PyMarkdown creates the correct HTML and correct Markdown from our parsed format.

That is where we still need our users to help us out. If you are scanning any Markdown documents and the results seem off, please file an issue. If you are starting to use our fix mode on your Markdown documents and there are issues, please file an issue. We appreciate any help that we can get to improve the project for everyone!

Added

• Issue 1212
  • added cases to Md031 for SetExt
  • added extra test cases and resolution to other cases

Fixed

• Issue 1169
• Issue 1170
• Issue 1173
• Issue 1175
• Issue 1176
  • fixed parser issue with extra leading space item being added
• Issue 1164
• Issue 1165
• Issue 1171
• Issue 1174
• Issue 1177
• Issue 1178
• Issue 1180
• Issue 1181
  • fixed rehydrator issue with extra leading space item being added
• Issue 1172
• Issue 1173
• Issue 1179
  • fixed rehydrator issue with reyhdrating after bqs with remove token being
    blank line
• Issue 1202
  • fixed rehydrator issue with paragraph continues
• Issue 1203
  • parser issue with fenced code block closing right away
• Issue 1204
  • rehydration/fix issue with blank lines causing off by one error
• Issue 1209
  • detection code for MD027 was not using the right container index
• Issue 1217
  • fixed issue with some list starts not being accounted for properly when
    grouped together on same line
• Issue 1166
• Issue 1167
• Issue 1168
  • fixed issue with more deeply nested containers

Changed

• Started to pull common code from MD031 for use in Md027

Version 0.9.23 - Date: 2024-09-04

This release continued our focus on enabling fixing for Rule Md031 and uncovering any issues with the more deeply nested container cases. And our luck held out, with the majority of the issues being related to the fixing algorithms. As mentioned in the last release, our detection rules rely on accurate parsing of the Markdown documents, with the only truthful way to verify that being to reconstitute the Markdown documents from our internal parsed format.

We continue to try different combinations of containers elements and leaf elements, verifying that PyMarkdown creates the correct HTML
and the correct Markdown from our parsed format. The good news is that the largest percentage of issues deal with how we represent and
reconstitute that whitespace. And we are diligently working to detect any issues with that process and to fix them.

That is where you, the users, come in. If you are scanning any Markdown documents and the results seem off, please file an issue. If you are starting to use our fix mode on your Markdown documents and there are issues, please file an issue. We appreciate any help that we can get to improve the project for everyone!

Added

• None

Fixed

• Issue 1141
  • fixed assert issue (test_extra_044mcv0)
• Issue 1142
  • fixed assert issue (test_extra_044lc)
• Issue 1143
  • fixed indent issue (test_extra_044ldb0)
• Issue 1144
  • fixed parsing issue (test_extra_044ldb1)
• Issue 1145
  • fixed indent issue (test_extra_044mx60)
• Issue 1146
  • fixed indent issue (test_extra_044lex1)
• Issue 1147
  • fixed indent issue (test_extra_044mcx)
• Issue 1148
  • fixed parsing issue (test_extra_044ldb1)
• Issue 1149
  • fixed parsing issue (test_extra_044mcz0)
• Issue 1150
  • fixed indent issue (test_extra_044lex3)
• Issue 1151
  • fixed assert issue with untested path (test_extra_044ldg)
• Issue 1152
  • fixed indent issue (test_extra_044mcs0)
• Issue 1153
  • fixed indent issue (test_extra_044mcu0)
• Issue 1154
  • fixed indent issue (test_extra_044mx31)
• Issue 1155
  • fixed indent issue (test_extra_044lde)
• Issue 1156
  • fixed indent issue (test_extra_044ldb0)

Changed

• None