EmEditor frequently uses the Registry to save and load settings (in order to reduce used memory size) when installed. However, when it is used as portable settings, it uses .INI files, and INI files are very slow compared to the Registry. For this reason, I advise using the Installer whenever possible. However, in most cases, both speed are about the same. The differences are significant when you change settings or when you first launch EmEditor.
Also, some features like Update Checker, File Extension Association with EmEditor, Explorer context menu, Windows 8/7 Taskbar context menu, etc., are only available from installed EmEditor.