First hackintosh with Windows dual boot using Intel NUC DC3217BY

wpid-2014-02-07-multibeast.png

My friend recently introduced me to the Intel NUC (DC3217BY). It’s basically a micro form factor barebone system that comes with Intel’s ULV i3 processor (powerful and low power consumpton). I decided to get one, slapped on 8 GB of ram, a 256 GB mSATA SSD, and a Broadcom based half-sized e-PCI network card, and Hackintosh’d it since the processor is similar to what’s in an Apple Macbook Air. Basic instructions for this particular hardware could be found here and here. A generic guide could be found here. For dual booting with Windows, this article and this post helped. This is what I recalled doing to set up:

  • Update the BIOS to the latest version
  • Create a bootable Windows 7 usb drive on Ubuntu using unetbootin (must be version 494) (drive must be formatted to NTFS)
  • Get Mac OS X 10.9.1 (Mavericks) from the Apple App store
  • Download Unibeast and Multibeast at tonymacx86
  • Download Chameleon Wizard
  • Download Kext Installer
  • On an existing machine with Mac OS X, /Applications/Utilities/Disk Utility and a (> 8 GB) usb flash drive to Mac OS Extended (Master Boot Record enabled).
  • Run Unibeast to load the Mac OS X installer on it
  • Copy Multibeast, Chameleon Wizard, and Kext Installer into this flash drive. Download DSDT.aml and the patched AppleIntelFramebufferCapri.kext here and place them on the flash drive as well (these are to get HDMI audio to work).
  • Boot up the flash drive, and boot the installer with the flags -x PCIRootUID=1 GraphicsEnabler=Yes per this post relevant to Mavericks
  • Once the Mac installer is booted, go to the Utilities Menu and launch Disk Utility. Format the hard drive into two partitions. The first should be called “Macintosh HD” and formatted to Mac OS Extended (Journaled) and the second should be called “BOOTCAMP” and formatted to MS-DOS (FAT).
  • Shutdown, insert in the Windows 7 usb, and install Windows on the second partition
  • Boot the Mac usb again, and install Mac OS X on the first partition
  • Boot the Mac usb again, and select to boot into the Mac OS X partition
  • Run Multibeast to do some post-configurations so that the hardware just works (options in image below)
  • Edit the org.chameleon.Boot via Chameleon Wizard per the image below
  • Copy DSDT.aml to /Extra/DSDT.aml
  • Install Kext Installer. Use it to install the patched AppleIntelFramebufferCapri.kext, then use it to rebuild permissions and kext cache. Restart the computer to have HDMI audio working.

Multibeast options: 2014-02-07-multibeast.png

Chameleon Wizard options for org.chameleon.Boot: 2014-02-07-chameleon-wizard.png

What would I do differently now? Consider getting a network card with bluetooth like the Dell DW1702 per this. I’m not sure if my monitor has speakers, so this would enable me to use wireless speakers. Update (8/22/2014): I ordered this wifi + bluetooth card (BCM943225 HMB/AzureWave AW-NB290) as it was cheap and that this guide shows it works well on a Mac after installing toledaARPT.kext from the repo. Now I have both wifi and bluetooth. The Windows driver for this card can be found here (direct link here).

Now, time to mount the small NUC to the back of my 27″ monitor.

Find text or string in files of a certain type

One can use grep "mystring" myfile.ext to find the lines in myfile.ext containing mystring. One could also use grep "mystring" *.ext to find mystring in all files with extension ext. Similarly, one could use grep "mystring" /directory to search for mystring in all files in the directory. What if one wants to search for mystring in all *.ext files in a certain path /directory? Most posts online would suggest something along the line of

<pre class="src src-sh">find /directory -type -f -name <span style="color: #ffa07a;">"*.ext"</span> | xargs grep <span style="color: #ffa07a;">"mystring"</span>

However, the comments of this post shows how one could do it with grep:

<pre class="src src-sh">grep -r --include=*.ext <span style="color: #ffa07a;">"mystring"</span> /directory

Guide to accessing MS SQL Server and MySQL server on Mac OS X

Native GUI client access to MS-SQL and MySQL

We can use Oracle SQL Developer with the jTDS driver to access Microsoft SQL Server. Note: jTDS version 1.3.0 did not work for me; I had to use version 1.2.6. Detailed instructions can be found here.

We can use MySQL Workbench to access MySQL server. Setup is intuitively obvious.

Overview of ODBC on Mac OS X

Mac OS X has iODBC installed as it’s default ODBC manager. Most other Linux/UNIX system uses unixODBC to manage the ODBC drivers. This is the main reason why there’s so much confusion on getting ODBC to work on Mac OS X.

ODBC is kind of like an API for any software to access any DBMS easily, regardless of what DBMS it is and what OS it’s running on. Different software (e.g., R or Python) can utilize ODBC to access different DBMS through the following logic: Software -> ODBC Manager -> ODBC Driver for the DBMS -> DBMS Server (Software: R, Python, etc.; DBMS: MySQL, MS-SQL, etc.).

It doesn’t matter whether you use iODBC or unixODBC. Whichever one you use, just make sure the DBMS Driver and software you are using are configured/compiled to use with the same ODBC manager (usually set through the configure flags). For example, the R package RODBC and Python package pyodbc are compiled by default to use iODBC on Mac OS X. The DBMS drivers used must be compiled for use with iODBC. For iODBC, one could add data source names (DSN’s) at ~/Library/ODBC/odbc.ini. For unixODBC, one could add DSN’s at ~/.odbc.ini.

My current setup utilizes iODBC. I will outline the instructions for setting up MySQL and freeTDS (MS-SQL) drivers for use with RODBC and pyodbc through iODBC.

MySQL and FreeTDS with iODBC on Mac OS X

Install the MySQL Connector/ODBC driver. Driver should be at /usr/local/lib/libmyodbc5.so or /usr/local/lib/libmyodbc5w.so. Note: I’m unable to compile the driver from source on Mac OS X.

FreeTDS is an open source ODBC driver to access MS SQL Server. Install via Home Brew:

## install homebrew
ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"

## install freetds
brew install freetds

Driver should be at /usr/local/lib/libtdsodbc.so (symbolic linked).

Create ~/Library/ODBC/odbc.ini:

[sqlserver01]
Driver=/usr/local/lib/libtdsodbc.so
TDS_Version=4.2
Server=ip.address
Port = 1433
Trace = Yes
Description=my description
# Database=
# can't specify username and password for freetds

[mysql01]
Driver=/usr/local/lib/libmyodbc5.so
Server=hostname
Port=3306
charset=UTF8 
User=username
Password=password
# Database=
## can specify an actual database to each DSN

Install pyodbc via sudo pip install pyodbc. Test connections in python:

import pyodbc as p

con1 = p.connect("DSN=sqlserver01;UID=username;PWD=password")
con1.execute("select name from master..sysdatabases").fetchall()

con2 = p.connect("DSN=mysql01;UID=username;PWD=password")
con2.execute("show databases;").fetchall()

Install R using the installer. Install RODBC in the R interpreter via install.packages("RODBC"). Test connections in R:

library(RODBC)

ch1 <- odbcConnect(dsn="sqlserver01", uid="username", pwd="password")
odbcQuery(ch1, "select name from master..sysdatabases")
odbcFetchRows(ch1)

ch2 <- odbcConnect(dsn="mysql01", uid="username", pwd="password")
odbcQuery(ch2, "show databases;")
odbcFetchRows(ch2)

More on unixODBC on Mac OS X

If one wants to use unixODBC on Mac OS X instead, note the following:

  • First install unixODBC via Homebrew with brew install unixodbc.
  • Compile R from source to have it work with unixODBC (R binaries from the installer uses iODBC by default).
  • Can choose --with-odbc-manager=odbc when compiling RODBC.
  • When compiling freeTDS, include the argument with-unixodbc (pass to Homebrew or when compiling manually).
  • I’m unable to compile the MySQL Connector driver on Mac OS X from source (Homebrew or manually). Thus, it won’t work with unixODBC. I believe I tried unixODBC and MySQL Connector from macports, and those work.
  • pyodbc only works with iODBC on Mac OS X (inspect setup file). Currently I can’t get pyodbc to work with unixODBC on Mac OS X.

More differences between unixODBC and iODBC

unixODBC comes with the isql command to access different DBMS from the command line interpreter. iODBC comes with the iodbctest and iodbctestw commands. The command isql works for me on Mac OS X when I set freeTDS up to work with unixODBC (e.g., accessing MS SQL Server). I couldn’t access MySQL server because the MySQL Connector driver was compiled for use with iODBC.

If I use iODBC, I get the following for trying to access a MySQL server:

$ iodbctestw "DSN=sqlserver01;UID=username;PWD=password"
iODBC Unicode Demonstration program
This program shows an interactive SQL processor
Driver Manager: 03.52.0607.1008
1: SQLDriverConnectW = [MySQL][ODBC 5.1 Driver]Prompting is not supported on this platform. Please provide all required connect information. (0) SQLSTATE=HY000
1: ODBC_Connect = [MySQL][ODBC 5.1 Driver]Prompting is not supported on this platform. Please provide all required connect information. (0) SQLSTATE=HY000

When I try to access SQL Server, I get

$ iodbctestw "DSN=sqlserver01;UID=username;PWD=password"
iODBC Unicode Demonstration program
This program shows an interactive SQL processor
Driver Manager: 03.52.0607.1008
1: SQLDriverConnectW = [FreeTDS][SQL Server]Login failed for user 'username'. (18456) SQLSTATE=42000
2: SQLDriverConnectW = [FreeTDS][SQL Server]Unable to connect to data source (0) SQLSTATE=08001
1: ODBC_Connect = [FreeTDS][SQL Server]Login failed for user 'username'. (18456) SQLSTATE=42000
2: ODBC_Connect = [FreeTDS][SQL Server]Unable to connect to data source (0) SQLSTATE=08001

Don’t know why that is so. I guess it’s not too important to use an interactive interpreter. What matter is that the driver works with R and Python. Perhaps I should consider sqsh or do more searching.

Compile R on Mac OS X

I did this before on my Macbook, but never documented. Wanted to document my attempt on the Mac virtual machine.

  1. Install Xcode (gcc/g++) and gfortran per this site.
  2. Download the R source file (2.12 in this example), and extract it.
  3. ./configure did not work (Fortran issue). Per this, this, and this, I used: ./configure --with-blas‘-framework vecLib’ –with-lapack –with-aqua –enable-R-framework F77=”gfortran -arch x86_64″= (the F77 is needed!).
  4. make then sudo make install.
  5. Add /Library/Frameworks/R.framework/Versions/Current/Resources/bin to PATH in ~/.bashrc.

Mac OS X in Linux via VirtualBox (as guest OS)

I recently installed a Windows virtual machine on my Linux laptop. I wanted to test out a virtualized Mac OS X machine via VirtualBox as well (just in case I ever needed to do something that can only be done on a Mac). My main references are this post and this post. Although they virtualize on a Windows host machine, it also worked on my Ubuntu 10.10 host laptop.

Things I needed:

  1. A Mac OS X 10.6 (snow leopard) installation dvd (RETAIL version, not the one that comes with the Apple machine). Buy a copy and rip it to iso.
  2. nawcom ModCD to be able to boot the installation DVD. The EFI that came with VirtualBox and the Empire EFI from the first post did not work.
  3. The latest myHack. I used version 1.1 (r123). Install this before the Apple update.
  4. Mac OS X 10.6.5 Update. Don’t do OS upgrade via the Automatic Updater, and don’t update without consulting the myHack website; things can break.

Some notes to myself:

  1. When creating the hard drive, use “dynamic” and just make it as big as possible. I originally did 20gigs and had to re-do to 100gigs. Since it is dynamic, you won’t take up space on your host system until you the space get filled up on the guest. Although there is a way to resize the hard drive on a Windows guest, I wasn’t able to do so on the Mac guest (different filesystem and there is that boot loader issue). Looking back in hindsight, I should’ve just made it 500gigs and move the virtualization on an external disk if I ever reach that (I highly doubt it!).
  2. To access files on the host, set up samba on the host and connect to it on the guest.
  3. I used dmg2img to convert dmg to iso (when needed).
  4. Follow the second post to change the resolution (modify /Extra/com.apple.Boot.plist and run =VBoxManage setextradata “MacOSX64″ “CustomVideoMode1″ “1366x768x32″= on the host). “1366x768x32″ is the resolution on my Toshiba Portege R705 running Ubuntu 10.10. Now I can go fullscreen.
  5. Sound most likely won’t work. I have no use for it anyways.

Good luck. I hope I won’t have to use it much, but it’s nice to do everything in one machine, running inside of Linux (as opposed to having multi-boot).

Migrating back to Linux (Ubuntu) from Mac OS X

After abour 1.5 year with Mac OS X on my beautiful black Macbook (v 4,1), I’ve decided to migrate back to Linux, namely Ubuntu 10.04 LTS. Why? Well, I was actually using Kubuntu (KDE version of Ubuntu) from about Spring 2006 to November 2008. At the time, I remember setting up Kubuntu on my Dell Inspiron 700m laptop was quite a task: getting the display to work at the right resolution, wireless, etc. Once everything was set up I loved the environment. Afew things bothered me then: no MS Office (OpenOffice doesn’t cut it sometimes when you have to collaborate), graphics environment (KDE) crashed from time to time (probably because laptop was too slow and I multi-tasked daily) and lack of support for third-party hardware (syncing Palm Centro and ipod, scanners, etc). I would have to do lots of research to figure out the third party issue, and often, a solution was not available (maybe arriving much later). I kind of got sick of this whole process so decided to switch to Mac OS X since it was Unix-based (all tools available), had MS Office, pretty, stable, nicely built (quality), and had third-party hardware support.

Once on the Macbook, I configured it to be my workstation with all the tools I need. Since Mac wasn’t truly Unix, compiling and getting certain software needed special attention despite the availability of MacPorts. Often times I would have to compile things from source manually (no apt-get!), maybe after having to tweak the source code a little.

Now I’m quite sick of not having apt-get or tweaking source code prior to compiling. So, I’m moving back. By now, the third-party issue isn’t too much of a concern since my ipod works with it and Android syncs to the cloud (and is Linux-based). Certain things will need to be resolved for new hardware specifications on newer Laptops, but this doesn’t bother me too much as I’m sure a solution will be available in a matter of weeks or months – plus, the community is very helpful. MS Office can run with WINE/Crossover Office (I did this before too, but I think things should be better now). I would also like to note that in the mist of all this I have servers and and a NAS that are debian- or ubuntu-based, and that I use the school’s Linux servers quite often. It’s only natural that my main workstation be Linux-based again, for ease of use and for efficiency and productivity.

using R + ess-remote with screen in emacs

Dear list,

I brought up this issue before but a good solution never arised: being able to use screen on a remote server (so if something goes wrong on my side I can always resume that R session) inside of emacs in order to utilize ESS. The closest thing I found to a good work flow was to use ansi-term or multi-term and copying and pasting code in Emacs (ESS only worked nicely with shell-mode; reason at the end). Some would advise to use screen and open emacs inside of that screen, and voila, you have the luxuries of screen (attaching the session anywhere) and emacs+ESS (keybindings, etc). However, I prefer to use one main emacs session on my laptop/netbook (all my configurations are there), where I can have multiple projects and multiple R jobs opened at once.

I would like to share what I have working (for the time being), with the help of Michael Zeller in case others are interested.

  1. Place the following in .emacs:
  <pre class="src src-sh">;; used to send screen keybindings to shell<span style="color: #00ffff;"> in</span> emacs

(define-key shell-mode-map (kbd “C-l”) (lambda (seq) (interactive “k”) (process-send-string nil seq))) (define-key inferior-ess-mode-map (kbd “C-l”) (lambda (seq) (interactive “k”) (process-send-string nil seq)))

  1. In xterm (or the likes), ssh to the remote server and start screen. Detach it. (Need to do this first as starting the initial screen in emacs shell-mode becomes very ugly with the printing; resuming the same screen session also becomes messy in xterm)
  2. In emacs, M-x shell. Set:
  <pre class="src src-sh">$ <span style="color: #eedd82;">TERM</span>=xterm <span style="color: #ff4500;">## </span><span style="color: #ff4500;">give clearing capabilities to shell-mode</span>

  1. ssh to remote server. screen -r to resume the screen session. start R. M-x ess-remote.
  2. Send R code from R source files like before!
  3. To detach or do anything screen related, precede EACH keybinding with C-l. For example, C-a C-d to detach will now be C-l C-a C-l C-d. Yes this is cumbersome, but I don’t imagine screen keybinding to be used much at this stage since we are developing and debugging R code for say a simulation study =].

I would also like to note (for archival reasons) that ess-remote does not work with ansi-term and multi-term because of the inferior-ess-mode command, which stems from comint-mode and inferior-ess-send-input. If you remove this command in ess-remote, you don’t get an error but u can only send one line of code at a time from the R file.

Hope this helps someone out there.

Stream music to client via DAAP

Now that I have a NAS at home, I plan to place stuff that takes up a lot of space that doesn’t get used so often on there. However, with that comes the issue of file access. Since my NAS is linux based – ubuntu server edition to be exact – I can access files via ssh, sftp, ftp, sshfs (map the drive), samba, etc. I should probably write a blog post about sshfs and samba soon.

However, what if I want to listen to music in my music player, say itunes on Mac/Windows or rhythmbox (or amarok, bashee, etc) on Linux? The good thing about itunes and these bloated music players are that they have a database of all your music files, so finding the song is fairly quick (type in artist name or song title). Double clicking the files on the mounted drive is no problem, but having this “search and play” capabilities is quite useful. Luckily, there are many solutions to this, one of which is the DAAP method. Read here for an overview.

Game plan is as follow:

  1. Set up music server on NAS.
  2. Be able to access the songs in itunes or the likes on remote servers in the same network.
  3. Be able to access the songs in itunes or the likes on remote servers when on a different network, say at school (or at work).

The first point is fairly easy on Ubuntu: install Firefly Music Server (formerly known as mt-daapd). Following these instructions, I did

 <pre class="src src-sh">sudo apt-get install mt-daapd

sudo apt-get install libid3tag0 sudo apt-get install avahi-utils ## needed for remote access sudo adduser media su media mkdir /home/media/music exit

Place music in /home/media/music or create a symlink to the music folder.

 <pre class="src src-sh">sudo /etc/init.d/mt-daapd start

sudo update-rc.d mt-daapd defaults

Now, from that computer, go to http://ip.address:3689 (username and password is mt-daapd). On this site, we can set up the configuration. Set username/password, password to listen, and music folder. Then scan the files. (Might have to do sudo /etc/init.d/mt-daapd start).

From here, computers on the network should be able to see the music server in itunes or rhythmbox (point 2) and play music (after entering music password). NOTE: I initially thought that you had to “Open Audio Stream…” or the likes on the music players, but that is for internet radio or something; DAAP servers will broadcast the music server to the network.

To do point 3, the first thing to do is set up port forwarding on the router (3689 to 3689 on the music server). The remaining part is a bit tricky. First, you have to do ssh tunneling so that the remote music server appears to be on the same network as your computer/laptop. Then we have to forward the “broadcast” from the DAAP server (the server that itunes sees, for example).

For a Linux client, instructions can be found here and here:

 <pre class="src src-sh">ssh my@daap.server.no-ip.org -N -f -L local.ip:6689:localhost:3689

avahi-publish-address -v -H “name of your host”.local -s “My daap share” _daap._tcp. 6689 & ## find name of host from echo $HOSTNAME

Now, itunes or other clients on the remote server should see “My daap share” on the Shared list. NOTE: currently, Linux players such as rhythmbox and banshee can’t play music this way. I’m sure a fix should be coming soon. If a mac is also on the same network as this current linux client, it will see the shared daap server and connect and play music.

For a Mac client, instructions can be found here:

 <pre class="src src-sh">dns-sd -P <span style="color: #ffa07a;">"Home iTunes"</span> _daap._tcp local 3689 localhost 127.0.0.1 <span style="color: #ffa07a;">"Arbitrary text record"</span> &amp;

ssh -C -N -f -L 3689:localhost:3689 myusername@blahblahblah.dyndns.org

Itunes should be able to see the remote daap server.

For a Windows machine, I’m sure one would have to rely on RendezvousProxy, following these instructions. I will test this out soon (setting this up for my girlfriend).