Here's a short tutorial i wrote up today that is a basic collection of Emacs Lisp Idioms.

The HTML version with links, syntax coloring, is at: http://xahlee.org/emacs/elisp_idioms.html

The text version follows. Comments, corrections, suggestions welcome.

(line endings in the following are eaten up by lj)


----------------------------- This page collects some basic emacs lisp programing patterns related to text processing.

This page is grouped into 2 sections: Interactive and Batch. The Interactive section focuses on idioms of writing the type of commands you call when actively editing. For example, looking up the word under cursor in google search, replace certain words in current region, insert XML template, rename a given function in the current programing project, etc. The Batch section focuses on batch style text processing, typically the type of tasks one would do in unix shell tools or Perl. For example, find and replace on a list of given files or dir, run XML validation on a bunch of files.

The idioms on this page contains only very basic ones, suitable for beginning elisp programer.

You should first be familiar with basic emacs functions that get cursor position, move cursor, search text, inserting and deleting text. See Elisp Common Functions Reference.

------------------------------ Interactive Command Idioms

Grabbing Text

Grab the text of given beginning position and ending position.

; get the string from buffer (setq myStr (buffer-substring myStartPos myEndPos)) (setq myStr (buffer-substring-no-properties myStartPos myEndPos))

Emacs's string can have properties for the purposes of syntax coloring, active button, hypertext, etc. The “buffer-substring-no-properties” function just return a plain string without these properties. However, most functions that takes string as argument can also accept strings that has properties.

Reference: Elisp Manual: Buffer-Contents.

------------------------------

Grabbing the current word, line, sentence, url, file name etc.

; grab a thing at point. The “thing” is a semantic unit. It can be a ; word, symbol, line, sentence, filename, url and others. (setq myStr (thing-at-point 'word)) ; grab the current word (setq myStr (thing-at-point 'symbol)) ; grab the current word with hyphens or underscore (setq line (thing-at-point 'line)) ; grab the current line

; grab the start and end positions of a line (or any other thing) (setq myBoundaries (bounds-of-thing-at-point 'line)) Note that, when the thing is a “symbol”, it usually means any alphanumeric sequence with dash “-” or underscore “_” characters. For example, if you are writing php reference lookup command, and the cursor is on p in “ print_r($y);”, you want to grab the whole “print_r” not just “print”. The exact meaning of symbol depends on the mode's Syntax Table.

Reference: Elisp Manual: Syntax-Tables.

------------------------------ Here's a example of php reference lookup command that grabs by “symbol”.

(defun php-lookup () "Look up current word in PHP ref site in a browser.\n If a region is active (a phrase), lookup that phrase." (interactive) (let (myword myurl) (setq myword (if (and transient-mark-mode mark-active) (buffer-substring-no-properties (region-beginning) (region-end)) (thing-at-point 'symbol))) (setq myurl (concat "http://us.php.net/" myword)) (browse-url myurl))) Reference: Elisp Manual: Buffer-Contents.

------------------------------ Grab Between Matching Pairs

Grab the current text between delimiters such as between angle brackets “<>”, parens “()”, double quotes “""”, etc.

(defun select-inside-quotes () "Select text between double straight quotes on each side of cursor." (interactive) (let (p1 p2) (skip-chars-backward "^\"") (setq p1 (point)) (skip-chars-forward "^\"") (setq p2 (point))

(goto-char p1) (push-mark p2) (setq mark-active t) ) )

------------------------------ Acting on Region

Idiom for a command that works on the current region.

Let your function have 2 parameters, such as “start” and “end”, then use “(interactive "r")”, then the parameters will be filled with beginning and ending positions of the region. Example:

(defun remove-hard-wrap-region (start end) "Replace newline chars in region by single spaces." (interactive "r") (let ((fill-column 90002000)) (fill-region start end)))

------------------------------ Idiom for acting on the region, if there's one, else, on the current word or thing.

(defun down-case-word-or-region () "Make current word or region into lower case." (interactive) (let (pos1 pos2) (if (and transient-mark-mode mark-active) (setq pos1 (region-beginning) pos2 (region-end)) (setq pos1 (car (bounds-of-thing-at-point 'word)) pos2 (cdr (bounds-of-thing-at-point 'word)))) (downcase-region pos1 pos2) ) )

------------------------------ Prompting and Getting Input

Idiom for promping user for input as the argument to your command.

Use this code “(interactive "‹code›‹promp string›")”. Example:

(defun query-friends-phone (name) "..." (interactive "sEnter friend's name: ") (message "Name: %s" name) )

What the “(interactive "sEnter friend's name:")” does is that it will ask user to input something, taken as string, and becomes the value of your command's parameter.

The “interactive” can be used to get other types of input. Here are some basic examples of using “interactive”.

“(interactive)” makes the function available thru interactive use, where user can call it with execute-extended-command (M-x).

“(interactive "s")” will prompt the user to enter a argument, taken as string, as argument to the function.

“(interactive "n")” will prompt the user to enter a argument, taken as number, as argument to the function.

The prompt text can follow the single-letter code string.

If your function takes multiple inputs, you can promp user multiple times, using a single call “interactive”, by joining the promp code string with “\n” in between, like this:

(defun query-friends-phone (name age) "..." (interactive "sEnter friend's name: \nnEnter friend's age: ") (message "Name: %s, Age: %d" name age) )

Reference: Elisp Manual: Defining-Commands.

------------------------------ Batch Style Text Processing Idioms

Open a file, process it, save, close it

; open a file, process it, save, close it (defun my-process-file (fpath) "process the file at fullpath fpath ..." (let (mybuffer) (setq mybuffer (find-file fpath)) (goto-char (point-min)) ;; in case buffer already open ;; do something (save-buffer) (kill-buffer mybuffer)))

For processing hundreds of files, you don't need emacs to keep undo info or fontification. This is more efficient:

(defun my-process-file (fpath) "process the file at fullpath fpath ..." (let () ;; create temp buffer without undo record. first space is necessary (set-buffer (get-buffer-create " myTemp")) (insert-file-contents filePath nil nil nil t) ;; process it ... (kill-buffer " myTemp")))

------------------------------ Find Replace strings:

; idiom for string replacement in current buffer; ; use search-forward-regexp if you need regexp (goto-char (point-min)) (while (search-forward "myStr1" nil t) (replace-match "myReplaceStr2")) (goto-char (point-min)) (while (search-forward "myStr2" nil t) (replace-match "myReplaceStr2")) ;; repeat for other strings ...

Calling a shell command.

; idiom for calling a shell command (shell-command "cp /somepath/myfile.txt /somepath")

; idiom for calling a shell command and get its output (shell-command-to-string "ls") Both shell-command and shell-command-to-string will wait for the shell process to finish before continuing. To not wait, use start-process or start-process-shell-command.

Reference: Elisp Manual: Asynchronous-Processes.

------------------------------ Traverse a directory recursively.

In the following, my-process-file is a function that takes a file full path as input. The find-lisp-find-files will generate a list of full paths, using a regex on file name. The “mapc” will apply the function to elements in a list.

; idiom for traversing a directory (require 'find-lisp) (mapc 'my-process-file (find-lisp-find-files "~/web/emacs/" "\\.html$"))

------------------------------

Xah ∑ http://xahlee.org/

☄

Effective Emacs

(Long term emacs productivity tips.)

Xah Lee, 2008-05-29

I have used emacs daily since 1998. Typically, i spent several hours inside emacs, everyday, for the past 10 years.

Here are 6 general emacs tips i felt that's the most important in emacs productivity, among all other emacs tips and tricks of my decade-long experience. If you use emacs only occasionally, these tips may not be very meaningful because they are general and does not solve any specific problems. If you have used emacs over a year, or is a dedicated emacs user, you might find these tips helpful.

perm url: http://xahlee.org/emacs/effective_emacs.html

following in a text version. (table formatting are screwed, no images, and no links)



------------------------------------------------- UNDERSTAND THE IMPORTANCE OF EMACS'S COMMANDS

In emacs, every keystroke is bound to a command (practically speaking). For example, when you type a key “e”, emacs actually runs the command self-insert-command. In other words, any key or key combination or key sequence you press is bound to a command. There are about 3000 commands in emacs. Most commonly used commands have a keyboard shortcut. For example, moving the cursor, opening file, copy paste, close a file, search text. To execute a command in emacs, type “Alt+x” followed by the command name.

The following will let you execute any command, or cancel it.

Command Name Shortcut Purpose execute-extended-command Alt+x Execute a given command keyboard-quit Ctrl+g cancel any key sequence or command in progress

The following will let you find out any command's name of a given shortcut, or a shortcut of a given command name.

Command Name Shortcut Purpose describe-key Ctrl+h k find out what command is associated with a shortcut describe-function Ctrl+h f find out what shortcut is bound to a function, and what the function do

The following will let you find out any features or shortcuts of a mode, and any command that you forgot the name.

Command Name Shortcut Purpose describe-mode Ctrl+h m read the inline doc of the current mode. For example, find out what features and shortcuts it provides. apropos-command Ctrl+h a list all commands who's name contains a given string.

Any operation in emacs is ultimately a command, and most of them are bound by a keyboard shortcut. By mastering the above commands, you will become self-sufficient.

Emacs provides a lot more other helping commands. For example, looking up variable, searching emacs manual, launching the emacs manual, searching commands with a given option, launching the emacs FAQ, launching emacs tutorial...etc, but the above i relied on for my first 5 years of using emacs, and still is most useful among all the help commands.

------------------------------------------------- MASTER WINDOW SPLITTING

Use split-window-vertically extensively, and give it a easy shortcut, such as “Alt+2” for spliting into 2 top and bottom panes, “Alt+1” to expand the pane the cursor is on, “Alt+0” to get rid of the current pane, and “Alt+o” for moving cursor to the next pane.

;;; WINDOW SPLITING (global-set-key (kbd "M-2") 'split-window-vertically) ; was digit-argument (global-set-key (kbd "M-1") 'delete-other-windows) ; was digit-argument (global-set-key (kbd "M-0") 'delete-window) ; was digit-argument (global-set-key (kbd "M-o") 'other-window) ; was prefix

In emacs, the window is split into panes frequently. It may be split by a shell-command's output, by various emacs grep command (e.g. list-matching-lines, grep-find), by emacs's various diff commands, by viewing man page (“Alt+x manual-entry”), by list-buffers, by viewing inline help (e.g. describe-function), by calc or calendar, or by lisp error messages. In many of the split pane situation, you can type “q” to exit the pane, but not always. And, some of the split panes put your cursor in the pane, but not all.

By mastering the above 4 generic window splitting commands, with easy-to-press shortcuts, you'll save a lot time opening or closing split panes. Also, by having a easy shortcut, now you can split and unsplit panes whenever you want and frequently. For example, i constantly have one pane showing dired and another pane showing a file content, or one pane of shell. A Top-Bottom-split window is extremely useful when coding. (for example, when i need to edit part of a file based on the current location, i often do a split pane first, then use isearch-forward to the location and edit, while viewing the original position.)

------------------------------------------------- MASTER DIRED

In coding, almost every hour you need to look at different files or directories, or do copying, deleting, renaming files or directories. Emacs provides a file management mode, called “dired”. (“dired” is acronym for DIRectory EDit. “Directory Editing” is 1980's term for file management.)

Dired is very useful. Once you master it, you will almost never use a graphical desktop. The only time i need to switch to OS's graphical desktop is when dealing with special files such as video, sound, images, etc. (Since emacs now can view images files too, so i almost don't go to desktop for image files now.) Also, emacs use the same interface of dired for ftp/sftp. So, for example, you can copy, move, delete, change file owner/perm, or edit files on remote servers.

For details about dired, see File Management with Emacs.

Also, in combination of dired, you should master the command “shell-command” (Alt+!) and “shell”. (For more detail about using shell commands, see Emacs and Unix Tips).

------------------------------------------------- REMAP MOST FREQUENTLY USED SHORTCUTS

The cursor moving commands are the most frequently used shortcuts. You use them every few seconds.

Emacs's default cursor moving shortcuts are “Ctrl+f”, “Ctrl+b”, “Ctrl+n”, “Ctrl+p”. The keys f, b, n, p are scattered around the keyboard and are not under the home row. Also, Control key is typed by the weak pinky finger. The Meta key (the Alt under thumb) is much easier to type. So, remap keys so that Alt with a home-row key moves the cursor.

;; make cursor movement keys under right hand's home-row. (global-set-key (kbd "M-j") 'backward-char) ; was indent-new-comment-line (global-set-key (kbd "M-l") 'forward-char) ; was downcase-word (global-set-key (kbd "M-i") 'previous-line) ; was tab-to-tab-stop (global-set-key (kbd "M-k") 'next-line) ; was kill-sentence

(global-set-key (kbd "M-SPC") 'set-mark-command) ; was just-one-space (global-set-key (kbd "M-a") execute-extended-command) ; was backward-sentence

For more extensive remapping, see: A Ergonomic Keyboard Shortcut Layout For Emacs.

------------------------------------------------- MASTER EMACS REGEX AND FIND-REPLACE COMMANDS

Searching text and find-replace text is tremendously useful. I use it just about every hour. (Many find-replace commands has a regex version) The following are the most useful search or find-replace commands:

Command Name Shortcut Target Description isearch-forward “Ctrl+s” cursor point to end (cycles back to file beginning) interactive search query-replace “Alt+%” region, or cursor point to end interactive find and replace query-replace-regexp “Ctrl+Alt+%” region, or cusor point to end interactive find and replace with regex pattern dired-do-query-replace-regexp In dired, “Q” multiple files interactive find and replace with regex pattern on multiple files

For detail about using these commands, in particular, how to control whether the search is case-sensitive, or whether the replacement is case-sensitive, see: Find and Replace with Emacs.

Many emacs find-replace commands uses a regex. Emacs has many other commands that uses regex. For example, list-matching-lines, delete-matching-lines, highlight-regexp, grep-find, dired-do-rename-regexp. Mastering emacs regex will be a good investment. Also, you should know how to enter TAB or Return character in emacs regex. (“\t” or “\n” does not work) For a short tutorial on most important tips of emacs's regex, see: Text Pattern Matching in Emacs.

------------------------------------------------- GET A GOOD KEYBOARD

You switch to different applications all day. Web browsers, emacs, terminal, Desktop, music player, image editor, ...etc. Their usage and interface changes, but there is one thing that does not change: Your keyboard.

Your keyboard is a intimate item. You touch it every minute. Emacs in particular, use modifier keys extensively. This may sound silly, but a good keyboard is one of the most important thing in productivity with emacs.

A good keyboard for emacs, should be one with large Alt and Control keys, and they should be available on both sides (one set for each hand, just like Shift key), and the right side's set should be positioned symmetrically (that is, the distance from your left index finger to the left Alt, should be the same to the distance from your right hand's index finger to the right Alt).

BAD

above: The Apple keyboard as of 2006. Note the ridiculous distance of the right side's modifier keys. It is not possible, to use the right thumb to press the alt key while the index finger remains on the J.

GOOD Microsoft Natural Multimedia keyboard

above: The Microsoft Natural Multimedia keyboard. Note, the keys are split and oriented for each hand. And, the Ctrl, Alt are very large and symmetrically positioned with respect to each hand's thumb. (See A Review of Microsoft Natural Keyboards)

Doesn't matter whether you like ergonomic keyboards, you should take a closer look at your keyboard, and see if it works with emacs well. Also, it's good to develop good habits when pressing the Control key. You should avoid pressing it with pinky, and should learn using alternate hand for Control-key combination, like you would with Shift key. For detail, see: How To Avoid The Emacs Pinky Problem.

------------------------------------------------- STEVE YEGGE'S EFFECTIVE EMACS

This article is inspired by Steve Yegge's Effective Emacs: http://steve.yegge.googlepages.com/effective-emacs. I like to thank Steve for his article.

However, I disagree with some of his tips strongly. In particular, his tips about Swapping Cap-lock and Control key, Invoking M-x without the Alt key, the “Lose the UI”. In the following, i give a brief explanation why i disagree.

Swapping Cap-lock and Control is not a good solution because it puts all the burden on the left pinky. The Control key on most PC keyboard is at the corner, and is very easy to press with palm. Also, there are 2 Control keys, on both sides. One should use them both like how Shift key is used, by using one hand for the modifier key and the other hand for the letter key. Alternating hands ease the repetitive burden, and avoids the awkward pinky-stretch. Swapping Cap-lock and Control can be a good solution on laptop keyboards. I think better is to actually swap Control and Alt. For detail, see How To Avoid The Emacs Pinky Problem and Why Emacs's Keyboard Shortcuts Are Painful.

Related to the above, i don't find the advice of “Invoking M-x without the Alt key”, by remapping to a Control key combination, a very good one. Personally, i remap M-x to M-a, since the “a” key is in the home row, and Meta (the Alt key under thumb) is easier to press then Control.

Steve advices users to “Lose the UI”. In general, i also work in emacs exclusively using keyboards. My emacs career, from 1998 to 2004 inclusive, are exclusive with text-terminals only (thru telnet/ssh on remote servers). Only after 2004, i started to use emacs in a graphical setting, under Mac OS X. I think graphical user interface is very helpful, because it gives a listing of the most useful commands, and can serve as a reminder or cheat-sheet. For example, i've been using dired for over 10 years. Some of dired features i hardly ever use. For example, looking at the dired “Operate” menu, i see that i've actually never used the shortcuts S, H, B, L, T, C-t D, C-t a, C-t t, C-t r. Under its “Mark” menu, i never used “* /”, “.”, “* *”, “M-{”, “M-}”. Similarly, there are many commands i've never used, or aware, that are listed under its Regexp, Immediate, Subdir menus. Maybe i've been missing out something, but emacs has 3000+ commands and lots of modes. Few people need to master all features. But occasionally, i can browse the graphical menu and find out the command that i kept forgetting to use, or see what most important commands are available for a new mode i just installed. I think graphical menus are quite useful in this aspect. Losing them doesn't save much screen space. I would agree that tool bar (the one with icons for opening file, copy/cut/paste, printing, help) is not much useful.

Texinfo Problems

Xah Lee, 2008-05

The Texinfo software produces invalid HTML documents. The following gives some detail.

In around 2006-09, i have created a HTML version of the GNU Emacs Lisp Reference Manual for my website, with corrections to the HTML and cleaned up CSS, so that the HTML are valid HTML documents and the CSS is handcrafted for better online presentation.

The version can be seen here: GNU Emacs Lisp Reference Manual. The story of my motivation is documented here (warning: rant): A Record of Frustration in IT Industry.

In the process of creating this cleaned up HTML version, there are several problems i found generated by the texinfo software. I thought i'd write a short summary of what they are, so that others who want to convert GNU docs to valid html might benefit, or that texinfo2html might fix these problems.

perm link: http://xahlee.org/emacs/texinfo_problems.html



Invalid HTML

Problems with texinfo generated html, with respect to html 4 transitional:

* there's no doctype declaration.
* when there's a footnote, it is generated as ‹p›‹hr›‹/div› which is invalid.

Problems with respect to html4 strict:

* “‹ol type=1 start=1›” should just be “‹ol›”.
* sometimes there's “‹/p›‹/blockquote›” but missing a opening “‹p›”.
* whenever there's a “‹b›Common Lisp note:‹/b›”, it should have a “‹p›” wrapped around the block, since it's inside “‹blockquote›” and html4strict requires it.

General HTML issues

* the css is plastered into every page. It should be one css file instead.
* it should declare utf8 as the charset. (so that it doesn't need to do a lot html character encoding)

Dead Links

In the elisp manual (one node per html page, roughly 850 html pages), there are 70 (local) links to other GNU documents. The local links are nice in that they provide cross-reference, but if one hosts only the elisp doc, all these local links will be dead.

Therefore, it would be nice, to have perhaps at texinfo level to embed markers to links that cross-ref to external docs, or perhaps at the html conversion level to provide a option to filter local links, so that local links can replaced as non-links (such as “See Emacs manual node on Abbrev”) or full http links to the right uri at gnu.org.

Vast majority of the 70 local links in the elisp doc are references to Emacs doc, but there are 6 that refers to widget, ses, cl, libc.

* problem link: elisp/Buttons.html ../widget/index.html
* problem link: elisp/Customization-Types.html ../widget/index.html
* problem link: elisp/Defining-New-Types.html ../widget/index.html
* problem link: elisp/Function-Safety.html ../ses/index.html
* problem link: elisp/Lisp-History.html ../cl/index.html
* problem link: elisp/Locales.html ../libc/Locales.html
* problem link: elisp/Time-Parsing.html ../libc/Formatting-Calendar-Time.html

I presume that people who wishes to host GNU doc do not want to host the entire set of GNU's documentation.
Use of ASCII

Also, the texinfo still use the convention of backtick ` and straight single quote ' to emulate curly ones “” and ‘’, and other ascii kludge such as “=›” instead of “⇒”. The ability to displaying these chars has been widely available on commercial platforms since mid 1990s, and on linuxes since about 2003 or so (emacs itself support unicode to a practical degree since emacs 21, released in 2001). It is perhaps time to update gnu doc convention to utf8 and use the proper characters.

This page shows you how to use emacs to do find and replace operations, and tells you how to do case-sensitive or case-insensitive match or replacement, and how to force captured regex text pattern into upper or lower case.

http://xahlee.org/emacs/emacs_find_replace.html

While readers here who use Emacs to post to LiveJournal using LJUpdate are probably already reading the ljupdate community, there may be some who aren't. Or there may be some who tried installing it, but encountered problems with the http-emacs project's elisp files and abandoned it.

The main problem with the http-emacs project is that it appears to be abandoned and none of the links to a complete downloadable package currently work. As I have already stated on my recent post to ljupdate.

So for those interested, I've made my own gzipped tarball and zip archive available for download.

Since the switch to Emacs 22.1, I've made a change in my VC practices for interfacing with CVS. I was disappointed to see that Dired-under-VC no longer seemed to show files that needed a patch or merge. I started using PCL-CVS and found that it had some very nice tricks up its sleeve. I was particularly impressed that the *cvs* buffer automatically updated itself as I made edits, compared to the need to hit the CVS repository to refresh a Dired-under-VC buffer.

So now that I'm all jazzed about PCL-CVS, I'm finally starting to work with a Subversion repository for one of my projects. Dired-under-VC does an OK job there, but nowhere near as good as PCL. Does anyone have any suggestions for better SVN interfacing?

This isn't an actually an issue with Emacs, but it's something I encountered when upgrading two of my machines to the latest version (22.1.1, which is very nice) and others might see from time to time.

Both installations worked fine, but when using X11 forwarding via SSH to the server I encountered the following error: "X protocol error: BadWindow (invalid Window parameter) on protocol request 38"

The issue is caused in OpenSSH 3.8 and later and is not related to the specific OS used. In my case I'm using a couple of different versions of Slackware, but most of the search results were discussing it in relation to the Mac OS, Tiger.

The solution is to use the SSH flag "-Y" for forwarding X11 connections from trusted hosts instead of the more traditional "-X" flag. So the command would be something like "ssh -Y [-l user | user@]hostname.example.com" instead of "ssh -X [-l user | user@]hostname.example.com" to have everything behave as intended.

Text Processing with Emacs Lisp
Xah Lee, 2007-10-29

This page gives a outline of how to use emacs lisp to do text
processing, using a specific real-world problem as example. If you
don't know elisp, first take a gander at Emacs Lisp Basics.

THE PROBLEM

Summary

I want to write a elisp program, that process a list of given files.
Each file is a HTML file. For each file, i want to remove the link to
itself, in its page navigation bar. More specifically, each file have
a page navigation bar in this format:

...

HTML version with links and colors is at:
http://xahlee.org/emacs/elisp_text_processing.html

how to stop livejournal from interpreting angle brackets as html tags?