Archive

Archive for the ‘CentOS’ Category

Fast tutorial to NLTK using Python. Example of Sentiment Analysis for movie reviews

2014/04/28 1 comment

Fast tutorial to NLTK using Python. Example of Sentiment Analysis for movie reviews

# http://www.nyu.edu/projects/politicsdatalab/workshops/NLTK_presentation%20_code.py
# http://www.nltk.org/howto/corpus.html

# We have python installed:

$ python
Python 2.6.6 ...

# And we try to use NLTK:

import nltk

	ImportError: No module named nltk

# It seems there is no nltk. Let's verify it:

import sys
for pth in sys.path:
    print pth

/usr/lib64/python26.zip
/usr/lib64/python2.6
/usr/lib64/python2.6/plat-linux2
/usr/lib64/python2.6/lib-tk
/usr/lib64/python2.6/lib-old
/usr/lib64/python2.6/lib-dynload
/usr/lib64/python2.6/site-packages
/usr/lib64/python2.6/site-packages/gtk-2.0
/usr/lib/python2.6/site-packages


# So, we need first to install nltk

exit()

# Go root:

$ yum install python-nltk
...Complete!

# Ok, let's try again:

$ python
Python 2.6.6 ...
import nltk
nltk.download()
d movie_reviews
	...
	Downloading package 'movie_reviews' to /home/jips/nltk_data...
	Unzipping corpora/movie_reviews.zip.
q

# We have now the corpus downloaded. Next step, use it:

import random
from nltk.corpus import movie_reviews
documents = [(list(movie_reviews.words(fileid)), category)
        for category in movie_reviews.categories()
        for fileid in movie_reviews.fileids(category)]
random.shuffle(documents)

all_words = nltk.FreqDist(w.lower() for w in movie_reviews.words())
word_features = all_words.keys()[:2000]
print word_features[:100]
	[',', 'the', '.', 'a', 'and', 'of', 'to', "'", 'is', 'in', 's', '"', 'it', 'that', '-', ')', '(', 'as', 'with', 'for', 'his', 'this', 'film', 'i', 'he', 'but', 'on', 'are', 't', 'by', 'be', 'one', 'movie', 'an', 'who', 'not', 'you', 'from', 'at', 'was', 'have', 'they', 'has', 'her', 'all', '?', 'there', 'like', 'so', 'out', 'about', 'up', 'more', 'what', 'when', 'which', 'or', 'she', 'their', ':', 'some', 'just', 'can', 'if', 'we', 'him', 'into', 'even', 'only', 'than', 'no', 'good', 'time', 'most', 'its', 'will', 'story', 'would', 'been', 'much', 'character', 'also', 'get', 'other', 'do', 'two', 'well', 'them', 'very', 'characters', ';', 'first', '--', 'after', 'see', '!', 'way', 'because', 'make', 'life']

def document_features(document):
    document_words = set(document)
    features = {}
    for word in word_features:
        features['contains(%s)' % word] = (word in document_words)
    return features

print document_features(movie_reviews.words('pos/cv957_8737.txt'))

featuresets = [(document_features(d), c) for (d,c) in documents]
train_set, test_set = featuresets[100:], featuresets[:100]
classifier = nltk.NaiveBayesClassifier.train(train_set)
print nltk.classify.accuracy(classifier, test_set)

	0.86

classifier.show_most_informative_features(5)

	Most Informative Features
		 contains(damon) = True              pos : neg    =     11.2 : 1.0
	   contains(outstanding) = True              pos : neg    =     10.6 : 1.0
		 contains(mulan) = True              pos : neg    =      8.8 : 1.0
		contains(seagal) = True              neg : pos    =      8.4 : 1.0
	   contains(wonderfully) = True              pos : neg    =      7.4 : 1.0

So, it works. Training a naive-Bäyes classifier with Python and NLTK library it is possible to find out what are most significant words that describe a good movie. This way, now it is easy to calculate a score of a movie comment and find out whether it is positive or negative. How do we call that? Sentiment Analysis.

Advertisements

How to fix failed dependencies when installing rpm Sphinx Search on CentOs

2014/02/15 6 comments

Trying to install the open source Sphinx Search Engine on my CentOs Linux server I got these errors:


wget "http://sphinxsearch.com/files/sphinx-2.1.5-1.rhel6.x86_64.rpm"
rpm -i sphinx-2.1.5-1.rhel6.x86_64.rpm

error: Failed dependencies:
libodbc.so.2()(64bit) is needed by sphinx-2.1.5-1.rhel6.x86_64
libpq.so.5()(64bit) is needed by sphinx-2.1.5-1.rhel6.x86_64

It means there are some libraries left in the system. To find out which are they I used this command:


yum provides **/libpq.so.5

This way I could install the missing libraries:


yum install unixODBC unixODBC-devel postgresql-libs

And then, I tried again (successfully this time) to reinstall Sphinx Search:


# rpm -i sphinx-2.1.5-1.rhel6.x86_64.rpm

Sphinx installed!
Now create a full-text index, start the search daemon, and you're all set.

To manage indexes:
editor /etc/sphinx/sphinx.conf

To rebuild all disk indexes:
sudo -u sphinx indexer --all --rotate

To start/stop search daemon:
service searchd start/stop

To query search daemon using MySQL client:
mysql -h 0 -P 9306
mysql> SELECT * FROM test1 WHERE MATCH('test');

See the manual at /usr/share/doc/sphinx-2.1.5 for details.

For commercial support please contact Sphinx Technologies Inc at
http://sphinxsearch.com/contacts.html

I hope this tip can help someone …

How to install PHP 6 on CentOS 6

2011/11/08 Leave a comment

How to install PHP6 on CentOS6: These are the Linux commands required to install latest PHP source code into a CentOS 6 LAMP server, and several useful libraries as: GD, Curl, Tidy, JSON, SQLite, PSpell, …


yum install wget
wget "http://snaps.php.net/php-trunk-201111071530.tar.bz2"
tar -jxvf php-trunk-201111071530.tar.bz2
cd php-trunk-201111071530
yum groupinstall "Development Tools"
yum install libxml2 libxml2-devel
yum install httpd-devel
yum install pcre pcre-devel
yum install bzip2 bzip2-devel
yum install gmp gmp-devel
yum install tidy libtidy libtidy-devel
yum install curl libcurl libcurl-devel
yum install libjpeg libjpeg-devel
yum install libpng libpng-devel
yum install libXpm libXpm-devel
yum install freetype freetype-devel
yum install aspell aspell-devel

./configure –build=x86_64-unknown-linux-gnu –host=x86_64-unknown-linux-gnu –target=x86_64-redhat-linux-gnu –program-prefix= –prefix=/usr –exec-prefix=/usr –bindir=/usr/bin –sbindir=/usr/sbin –sysconfdir=/etc –datadir=/usr/share –includedir=/usr/include –libdir=/usr/lib64 –libexecdir=/usr/libexec –localstatedir=/var –sharedstatedir=/var/lib –mandir=/usr/share/man –infodir=/usr/share/info –cache-file=../config.cache –with-libdir=lib64 –with-config-file-path=/etc –with-config-file-scan-dir=/etc/php.d –disable-debug –with-pic –disable-rpath –without-pear –with-bz2 –with-exec-dir=/usr/bin –with-freetype-dir=/usr –with-png-dir=/usr –with-xpm-dir=/usr –enable-gd-native-ttf –without-gdbm –with-gettext –with-gmp –with-iconv –with-jpeg-dir=/usr –with-openssl –with-pcre-regex=/usr –with-zlib –with-layout=GNU –enable-exif –enable-ftp –enable-magic-quotes –enable-sockets –enable-sysvsem –enable-sysvshm –enable-sysvmsg –with-kerberos –enable-ucd-snmp-hack –enable-shmop –enable-calendar –with-sqlite3 –with-libxml-dir=/usr –enable-xml –with-system-tzdata –with-apxs2=/usr/sbin/apxs –with-mysql –with-gd –disable-dom –disable-dba –without-unixODBC –disable-pdo –disable-xmlreader –disable-xmlwriter –disable-phar –disable-fileinfo –enable-json –with-pspell –disable-wddx –with-curl –disable-posix –disable-sysvmsg –disable-sysvshm –disable-sysvsem –enable-mbstring –with-mysqli –with-tidy

make
make install
sed -i ‘/^LoadModule php5_module/ s/^/#/’ /etc/httpd/conf/httpd.conf
service httpd restart

Easy install MySQL on CentOS 6

2011/08/08 1 comment
CentOS

Install MySQL on CentOS 6

Assuming you have installed Linux CentOS 6 and network is running correctly, follow next commands to install MySQL server. I did run these commands on a fresh Full Desktop CentOS 6 (64 bits) install over VM VirtualBox. Nevertheless, the following commands will be useful also on other configurations. Use root permissions.

yum install mysql mysql-server
service mysqld start
chkconfig --levels 235 mysqld on
/usr/bin/mysql_secure_installation

mysql -u root -p
	CREATE DATABASE mydb;
	CREATE USER 'mydbuser'@'*' IDENTIFIED BY 'mydbpass';
	GRANT ALL ON mydb.* TO mydbuser@'*';
	FLUSH PRIVILEGES;

// Granting remote access to MySQL Server on port 3306:
vi /etc/sysconfig/iptables
// Write this line before last line "COMMIT"
-A INPUT -m state --state NEW -m tcp -p tcp --dport 3306 -j ACCEPT

service iptables restart

Done! Now try to connect to your database.