[Patrick C]

I’ve now spent some time thinking about this problem.

I do have an EmEditor macro that can calculate the Levenshtein distance / similarity between two strings.
Its adapted from
[1] https://github.com/gustf/js-levenshtein

The problem is what you have already stated:

macro that would process a document with tens of millions of rows and display the similarity percentage in a separate column […] would be very slow using the Levenshtein algorithm.

What EmEditor can do is sort all lines by similarity with any given line.
I.e. just one line:
E.g. line 1 with 2, 3, 4, 5, … 1’000’000 (one million lines example)
One could then, for example, write a macro to calculate the similarity of the first 100 or 1000 lines, i.e. the lines that are now the most similar, with line 1 and display the similarity percentage in a separate column.
↳ I’ve already written a proof of concept hack of such a macro.
↓
The problem is that you will then only know the similarity of line 1 with the most similar 1000 or so other lines.
Should you need to then compare
Line 2 with all others
Line 3 with all others
etc. up to line 1’000’000
would probably require a massively powerful computer and on top of that using EmEditor would no longer make sense.

So for your question

I want to know how I can initially filter the database using the program’s [EmEditor’s] powerful tools.

To filter the database you would first need to know the similarity percentage. And once the similarity percentage is known, pre-filtering would no longer be necessary as the final result is already there.

Hope this helps.

[Patrick C]

So I want to know how I can initially filter the database using the program’s powerful tools.

Perhaps.

—————————
How do you want to compare the rows?
E.g.
[Case 1] Compare adjacent rows 1,2 3,4 5,6

Or
[Case 2] Each row with all other rows?
1 with 2, 3, 4, 5, …
then 2 with 3, 4, 5, …
——————

I’m assuming your requirement is closer to [Case 2].
EmEditor can sort by similarity.
What one could do is sort by similarity with EmEditor’s built in algorithm (hopefully fast) and then use Javascript to calculate the similarity between adjacent lines, i.e.
1 with 2,
2 with 3
3 with 4
etc.

[Patrick C]

Yes, the result is the same as eePromptMultiline only applies to the string input box.

prompt(“● Line 1/3\n● Line 2/3\n● Line 3/3”, “”, eePromptMultiline);

As before, the message’s
● Line 2/3
● Line 3/3
are not visible.

[Patrick C]

This sounds like an application for an AI, i.e. EmEditor’s ChatAI plugin.
In principle you can ask the AI to categorise the strings in EmEditor’s current document, add a tab separated column showing the similarity, paste the result into a new EmEditor document and display it in tab separated CSV mode.

Should you want to use an AI to perform this task, then my guess is that you might require an AI consultant, rather than an EmEditor consultant.

[Patrick C]

Sort-Object works in my case (I OCR copy pasted line 47 of your example)

In Tools → Properties for Current Configuration → Highlight (1)

What you can check is:

1) When importing the esy file with “Import…”:
Select “Yes” to remove old items. → Prevent conflict with previous definitions.

2) In the drop down list just below the “Enable Keyword Highlight” drop down box:
Select “Only User-Defined Strings”

[Patrick C]

My guess:
Line 163 contains “write”
“Write-Host” is on line 537, resulting in “write” matching before “Write-Host”.
However if this were true, then none of the other “Write-…” statements would match.
What you can try is test this by deleting line 163.

In case of interest:
While its not perfect, I do have an alternative PowerShell highlighter file.
PowerShell_by_PC.esy

[Patrick C]

I’m glad the macro helped.
Thank you for fixing the issue!

[Patrick C]

Hello Yutaka,

I wrote a macro that triggers the error:
hDataNull_error_demo.jsee

EmEditor 25.3.910 still has the issue.

Settings:
● Virtual Space is enabled.
● In the find dialogue: Treat CR and LF separately must be enabled.
● Encoding: UTF-8 without BOM
● Newline character: “LF”

Many thanks!

[Patrick C]

I’ve now figured out what triggers the
Text copy error: “hData = NULL”
behaviour:
Search for text using the find dialogue:

[Patrick C]

Version 25.3.909
Text copy error: “hData = NULL”

Difficult to reproduce text copy error:
● When freshly starting EmEditor, e.g. close and open:
→ The text copy error does not immediately occur.
● Its unclear what triggers the behaviour.

Text copy behaviour once “triggered”:

PS Thank you for the last fix (markers)!

[Patrick C]

Version 25.3.906
Markers don’t update when changing them in the Customise → Markers dialogue

PS Thanks for fixing the toolbar configuration bug back in August 2025.

[Patrick C]

Its a bit weird to click “Open” when saving, but I think I can live with this.
Thank you!

[Patrick C]

I don’t want to use an external downloader, wget or any other tool except the macro.

I’d be surprised if this were possible without any external tool.
If I’d write the macro and not want to install an external tool, I’d use EmEditor’s shell.Run(…) command together with Windows Powershell’s Invoke-WebRequest … -OutFile … command.
PowerShell is built into Windows so there should be no need to install it.

[Patrick C]

just the file name without the extension

To my knowledge EmEditor has not got a document name without extension property.

In JavaScript I’d do the following:

let bareName = "";
let iLastIdx = document.Name.lastIndexOf(".");
if (iLastIdx > 0) {
  bareName = document.Name.substring(0, iLastIdx);
}
OutputBar.writeln("bareName = «" + bareName + "»");

[Patrick C]

Always glad to help :)

[Patrick C]

Its document.FullName rather than document.FileName.
file:///C:/ProgramData/Emurasoft/EmEditor/Help/en/macro/document/document_fullname.html

[Patrick C]

I’ve adapted the list bookmarks macro for this task:
List all finds by repeating the most recent search.
Usage:
1) Find any match using EmEditor’s find dialogue.
2) Run the macro.
The macro will then list all occurrences in the output bar.

Example output

The macro is configurable:
● Find in all open documents.
● List the entire line Y/N.
● List the find Y/N.
● …

[Patrick C]

I’m glad it helps and thank you for the feedback :)

The other navigation option would be to use the outline bar with custom regex.
But getting the regex right can be a pain and only the current document is displayed.

[Patrick C]

(?#_text_c==0)"[^/]*?"
Isn’t exactly a fix as it makes it impossible for the string to contain a /.

Side note:
The regex I use in the example are intentionally simplified for the sake of illustration.
The regex to match a string actually is (?#_text_c==0)".*?(?<!\\)", which in the example is simplified to (?#_text_c==0)”.*?”.

[Patrick C]

Yes, thank you for asking!
While the formatting is a lot better, there is the following shortcoming:

Rather than not applying a rule,
(?^#_text_c==0) only postpones a rule’s formatting until c==0.
This can lead to incorrect formatting.

Example case (simplified regex):
Rule 1) Format javascript strings (?^#_text_c==0)".*?"
and
Rule 2) Format javascript template literals (?^#_text_c==0)\/.+?\/

On line 2:
The formatting between the «; "» is incorrect.
And rule 1 is not applied to «"a string"»

If it were possible to set (?^#_text_c==0) to ignore (i.e. not apply) the rule rather than just postpone its formatting, then this shortcoming would be solved.

[Patrick C]

You could display them in the OutputBar.

I’ve adapted a script of mine with respect to listing bookmarks:
list_bookmarks_in_the_output_bar.jsee

Example output:

Perhaps this helps.

[Patrick C]

Thank you! 🙏
I’ve just adapted my JavaScript highlighter template and the results are fantastic:

String and regex literals now render really well 😃.
With respect to single line highlighting EmEditor now is perfect for my needs.

The only thing I do not have a solution for is multiline matching.
As an example: For the JavaScript multiline comment /*…*/ one could use the regex
(?#_text_c==0)\/\*.*?\*\/ with the /s flag:

Should something like a /s flag or a directive
#Keyword color=10, …, regexp=on, multiline=on
be possible, then EmEditor’s highlighter would be one of the best I’ve ever seen.

[Patrick C]

Essentially I need the following (using the regex matching example at the top):

(?#_text_c==0)″.*?″ 
(?#_text_c==0)%%.*$

I.e. apply the highlight only when the start of the match uses
SMART_COLOR_NORMAL = 0

This works fantastically well, but only on odd lines:

→ Line 1, 3, 5 and 7 are correct.

My test file is UTF-8 with LF as line terminator (no CR).

I cannot thank you enough for taking time for this!

[Patrick C]

Wow 😃
I’ll test this tomorrow Friday and will give feedback.
Thank you very much Yutaka!

[Patrick C]

The %% is just for the sake of illustration.

I came up with the idea of using regex expressions for highlighting from other highlighters.

In the case of highlighting javascript these are:
[1] https://github.com/pygments/pygments/blob/master/pygments/lexers/javascript.py
[2] https://github.com/speed-highlight/core/blob/main/src/languages/js.js
and several others.

To highlight javascript regex literals, /…/ one can, for example use (from [1]):
\/((?!\/)[^\r\n\\]|\\.)+\/[dgimsuy]*
And to highlight template literals ‵…‵
‵(?:(?!‵|${).)*?(?:‵|\${)
}(?:(?!‵|${).)*?(?:‵|\${))

The problem is that these two interfere:

I’ve written several variable length regex based highlighters for Python, Javascript, PowerShell and more. These work well, but only up to the point where they don’t overlap with another variable length regex highlight definition.

I’ll consider adding an option to control how regex is applied in situations like this.

This would be awesome 😃.
I realise that I’m just one customer, so please first focus on what’s most important for EmEditor rather than my request. Should you find the time, then I’ll greatly appreciate the effort.
Thank you, Yutaka!

[Author]

Posts