unexpected[it]

Professional IT Blog

  • 29th, Jul 2010

MySQL Cluster on Amazon EC2

Tags: , , , , , ,

I created two cluster servers and one management server for MySQL Cluster

AMI: Minimal Fedora Core 8, 32-bit architecture, Apache 2.0, and Amazon EC2 AMI Tools
MySQL Cluster version 7.1
Fedora Version: 8
Security Group: default

1) Download RPM files
From http://www.mysql.com/downloads/cluster/#downloads all rpm files for Red Hat & Oracle Enterprise Linux for (x86, 32-bit).

2) In each instance

yum --enablerepo=remi erase mysql mysql-libs mysql-server mysqlclient15.i386
rpm -iv MySQL-Cluster-gpl-*

3) Config Management Server
Server IP ( ec2-79-11-111-11.eu-west-1.compute.amazonaws.com )

vi /var/lib/mysql-cluster/config.ini
[NDBD DEFAULT]
NoOfReplicas=2
[MYSQLD DEFAULT]
[NDB_MGMD DEFAULT]
[TCP DEFAULT]
# Managment Server
[NDB_MGMD]
HostName=ec2-79-11-111-11.eu-west-1.compute.amazonaws.com		# the IP of THIS SERVER
# Storage Engines
[NDBD]
HostName=ec2-79-22-222-22.eu-west-1.compute.amazonaws.com		# the IP of the FIRST SERVER
DataDir= /var/lib/mysql-cluster
[NDBD]
HostName=ec2-79-33-333-33.eu-west-1.compute.amazonaws.com		# the IP of the SECOND SERVER
DataDir=/var/lib/mysql-cluster
# 2 MySQL Clients
# I personally leave this blank to allow rapid changes of the mysql clients;
# you can enter the hostnames of the above two servers here. I suggest you dont.
[MYSQLD]
[MYSQLD]

ndb_mgmd -f /var/lib/mysql-cluster/config.ini

The Management Server is now ready, you can check the conexions with the other servers doing this

ndb_mgm
ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)]	2 node(s)
id=2	@79.22.222.22  (mysql-5.1.44 ndb-7.1.4, Nodegroup: 0, Master)
id=3	@79.33.333.33  (mysql-5.1.44 ndb-7.1.4, Nodegroup: 0, Master)

[ndb_mgmd(MGM)]	1 node(s)
id=1	@79.11.111.11  (mysql-5.1.44 ndb-7.1.4)

[mysqld(API)]	2 node(s)
id=4	@79.22.222.22  (mysql-5.1.44 ndb-7.1.4)
id=5	@79.33.333.33  (mysql-5.1.44 ndb-7.1.4)

4) Config Cluster Servers
Cluster Server IPs: (ec2-79-22-222-22.eu-west-1.compute.amazonaws.com and ec2-79-33-333-33.eu-west-1.compute.amazonaws.com )

/etc/my.cnf
[mysqld]
ndbcluster
ndb-connectstring=ec2-79-11-111-11.eu-west-1.compute.amazonaws.com     # the IP of the MANAGMENT SERVER
[mysql_cluster]
ndb-connectstring=ec2-79-11-111-11.eu-west-1.compute.amazonaws.com	# the IP of the MANAGMENT SERVER

/usr/sbin/ndbd --initial

/etc/init.d/mysql start

5) Testing
You can check in FIRST or SECOND server this and it will be duplicated in both servers.

mysql -u root -p
use test;
CREATE TABLE ctest (i INT) ENGINE=NDBCLUSTER;
INSERT INTO ctest () VALUES (1);
SELECT * FROM ctest;

6) Starting and Shutting down the MySQL Cluster
Starting
Management Server: db_mgmd -f /var/lib/mysql-cluster/config.ini
Cluster Servers: ndbd

Shutting down
Management Server: ndb_mgm> shutdown

Notes
Error: ERROR 1005 (HY000): Can’t create table ‘test.ctest’ (errno: 157)
Try to track the error with SHOW WARNINGS; When I have this problem I had disabled SELinux with “setenforce 0″.

Error: There is no populating changes trough the cluster servers.
Make sure they are connected each other, you can check this from management server with ndb_mgm> SHOW
Also double check all servers are in the security group “default” to leave the ports open. (see image)

Pages related: http://dev.mysql.com/tech-resources/articles/mysql-cluster-for-two-servers.html

  • 29th, Jul 2010

Setting up a Magento Server in Amazon Cloud EC2

Tags: , , , , , , , , , ,

How to create an instance for Magento in Amazon Cloud EC2

You can also get this ebs Image in Amazon EC2. Choose “426063643107/Magento 1.4.x on Fedora 8″
when you launch an instance in the tab “Community AMIs”
MySQL Admin
Username: root Password: 123456

Once the image is initialized
a) Install Magento 1.4.1.1 (For demo store follow 5.1)
In your browser go to: http://YOUREC2NAME.compute-1.amazonaws.com/magento/
b) Install Magento 1.4.1.1 DEMO Version
In your browser go to: http://YOUREC2NAME.compute-1.amazonaws.com/magento-demo/

1) Choose the simplest Instance Fedora Core 8 (AMI Id: ami-df1e35ab)
Minimal Fedora Core 8, 32-bit architecture, Apache 2.0, and Amazon EC2 AMI Tools.

2) Installing Remi Repository with Apache, MySQL and PHP

wget http://rpms.famillecollet.com/remi-release-8.rpm
rpm -Uvh remi-release-8.rpm
vi /etc/apt/sources.list.d/remi.list # Uncomment: repomd http://rpms.famillecollet.com/ fc$(VERSION).$(ARCH)

# Installing MySQL
#
yum --enablerepo=remi install mysql mysql-server mysql-devel

# Installing Http Server with PHP
#
yum --enablerepo=remi install php php-mysql php-common php-gd php-mbstring php-mcrypt php-devel php-xml

# Config MySQL
#
/usr/bin/mysqladmin -u root password '123456'

# Config Apache
#
# /etc/httpd/conf/httpd.con on Line 326 change
AllowOverride None
# to
AllowOverride All
# inside <Directory "/var/www/html"> ... </Directory>

# /etc/php.ini Activate this line to read <? tags
short_open_tag = On

#Try it out with the usual script in /var/www/html/test.php
&lt;?php
phpinfo();
?>

3) Installing Exim Mail Server

yum install exim
/etc/init.d/exim start
mail -s "testing" your@emailaddress.com
Testing purpose!
.
Cc:
# Your Mail Server should be running properly

4) SFtp Server Access

# Just using a SFtp Client Server Vicomsoft Ftp Client
#  configure a new connection without password and use
#  the option "Use SSH keyfile (SFTP only)" to add your
#  .pem file
#  Or just use
sshfs -i MYSECRETKEY.pem root@YOUREC2NAME.compute-1.amazonaws.com:/ /

5) Using ntsysv command choose the next services to start automatically
sendmail
httpd
mysqld

6) Optionally you can give NFS support to share files between images

#Server1
[root@server1]# vi /etc/exports
/var/www/html *(rw)

[root@server1]# /etc/init.d/rpcbind start

[root@server1]# /etc/init.d/nfs start
#Server2
[root@server2]# mount -o nolock ec2-79-xxx-xx-xxx.eu-west-1.compute.amazonaws.com:/var/www/html /mnt

7) Download and uncompress Magento versions in /var/www/html

CHANGELOG
File /etc/php.ini change short_open_tag = On

  • 24th, Jul 2010

Magento Useful Methods

Tags: , , ,

//Get Store Configuration Setting
echo Mage::getStoreConfig("carriers/freeshipping/free_shipping_subtotal");

//Get Currency Symbol
echo Mage::app()->getLocale()->currency(Mage::app()->getStore()->getCurrentCurrencyCode())->getSymbol();

//Print Amount Proper Currency (e.g. $102.36)
echo Mage::helper('core')->currency($subtotal, true, true)
  • 24th, Jul 2010

Speed up images and flash loading times in sites

Tags: , , , ,

.htaccess file


Header set Cache-Control “max-age=7257600, public”

It means that image files are all forced to use the local cache for 3 months.
Speeds many sites up considerably.

  • 23rd, Jul 2010

Magento Customer by Orders Total Report PHP Script

Tags: , , , , , , , ,

mysql_connect("localhost","root","root", 3306) or die(mysql_error());
mysql_select_db("MagentoDatabase") or die(mysql_error());
//Get Customer Entity Type Id
$query = "SELECT entity_type_id FROM `eav_entity_type` WHERE entity_type_code = 'customer'";
$query_Row = mysql_query($query);
$qs =  mysql_fetch_assoc($query_Row);
$customerEntity = $qs["entity_type_id"];
//Get Customer Entity Address Id
$query = "SELECT entity_type_id FROM `eav_entity_type` WHERE entity_type_code = 'customer_address'";
$query_Row = mysql_query($query);
$qs =  mysql_fetch_assoc($query_Row);
$customerAddressEntity = $qs["entity_type_id"];

//Get Attributes Ids For Customer
$attributes = array("'email'", "'prefix'", "'firstname'", "'lastname'", "'default_billing'");
$query = "SELECT attribute_code, attribute_id FROM eav_attribute WHERE attribute_code IN ( ".implode(",", $attributes)." ) AND entity_type_id = ".$customerEntity;
$query_Row = mysql_query($query);
while($qs =  mysql_fetch_assoc($query_Row)){
	$attributeIds[$qs["attribute_code"]] = $qs["attribute_id"];
}

//Get Attributes Ids For Customer Address
$attributes = array("'company'", "'street'", "'postcode'", "'city'", "'region'", "'country_id'", "'telephone'");
$query = "SELECT attribute_code, attribute_id FROM eav_attribute WHERE attribute_code IN ( ".implode(",", $attributes)." ) AND entity_type_id = ".$customerAddressEntity;
$query_Row = mysql_query($query);
while($qs =  mysql_fetch_assoc($query_Row)){
	$attributeIds[$qs["attribute_code"]] = $qs["attribute_id"];
}

$query = "
SELECT e.customer_id AS customer_id, ROUND(SUM((e.base_subtotal-IFNULL(e.base_subtotal_refunded,0)-IFNULL(e.base_subtotal_canceled,0))*e.base_to_global_rate), 2) AS orders_sum_amount, ROUND(AVG((e.base_subtotal-IFNULL(e.base_subtotal_refunded,0)-IFNULL(e.base_subtotal_canceled,0))*e.base_to_global_rate),2) AS orders_avg_amount, COUNT( e.entity_id ) AS  `orders_count` ,  `_table_email`.`email`, `_table_prefix`.`value` AS `prefix`, `_table_firstname`.`value` AS `firstname`, `_table_lastname`.`value` AS `lastname`, `_table_default_billing`.`value` AS `default_billing`, `_table_billing_company`.`value` AS `billing_company`, `_table_billing_street`.`value` AS `billing_street`, `_table_billing_postcode`.`value` AS `billing_postcode`, `_table_billing_city`.`value` AS `billing_city`, `_table_billing_region`.`value` AS `billing_region`, `_table_billing_country_id`.`value` AS `billing_country_id`, `_table_billing_telephone`.`value` AS `billing_telephone`
FROM  `sales_order` AS  `e`
INNER JOIN `customer_entity_varchar` AS `_table_prefix` ON (_table_prefix.entity_id = e.customer_id) AND (_table_prefix.attribute_id=".$attributeIds['prefix'].")
INNER JOIN `customer_entity_varchar` AS `_table_firstname` ON (_table_firstname.entity_id = e.customer_id) AND (_table_firstname.attribute_id=".$attributeIds['firstname'].")
INNER JOIN `customer_entity_varchar` AS `_table_lastname` ON (_table_lastname.entity_id = e.customer_id) AND (_table_lastname.attribute_id=".$attributeIds['lastname'].")
INNER JOIN `customer_entity_int` AS `_table_default_billing` ON (_table_default_billing.entity_id = e.customer_id) AND (_table_default_billing.attribute_id=".$attributeIds['default_billing'].")
LEFT JOIN `customer_address_entity_varchar` AS `_table_billing_company` ON (_table_billing_company.entity_id = _table_default_billing.value) AND (_table_billing_company.attribute_id=".$attributeIds['company'].")
LEFT JOIN `customer_address_entity_text` AS `_table_billing_street` ON (_table_billing_street.entity_id = _table_default_billing.value) AND (_table_billing_street.attribute_id=".$attributeIds['street'].")
LEFT JOIN `customer_address_entity_varchar` AS `_table_billing_postcode` ON (_table_billing_postcode.entity_id = _table_default_billing.value) AND (_table_billing_postcode.attribute_id=".$attributeIds['postcode'].")
LEFT JOIN `customer_address_entity_varchar` AS `_table_billing_city` ON (_table_billing_city.entity_id = _table_default_billing.value) AND (_table_billing_city.attribute_id=".$attributeIds['city'].")
LEFT JOIN `customer_address_entity_varchar` AS `_table_billing_region` ON (_table_billing_region.entity_id = _table_default_billing.value) AND (_table_billing_region.attribute_id=".$attributeIds['region'].")
LEFT JOIN `customer_address_entity_varchar` AS `_table_billing_country_id` ON (_table_billing_country_id.entity_id = _table_default_billing.value) AND (_table_billing_country_id.attribute_id=".$attributeIds['country_id'].")
LEFT JOIN `customer_address_entity_varchar` AS `_table_billing_telephone` ON (_table_billing_telephone.entity_id = _table_default_billing.value) AND (_table_billing_telephone.attribute_id=".$attributeIds['telephone'].")
INNER JOIN `customer_entity` AS `_table_email` ON (_table_email.entity_id = e.customer_id)
WHERE (
e.entity_type_id =  '11'
)
AND (
e.state !=  'canceled'
)
AND (
e.created_at >=  '".date('1900-01-01 00:00:00')."'
AND e.created_at <=  '".date('Y-m-d H:i:s')."'
)
GROUP BY  `e`.`customer_id`
ORDER BY orders_sum_amount DESC
";
$query_Row = mysql_query($query);
$i=0;
echo "<pre>";
//Headers
$headers = array("customer_id", "email", "orders_sum_amount", "orders_avg_amount", "orders_count", "prefix", "firstname", "lastname", "default_billing","billing_company","billing_street1","billing_street2","billing_street3","billing_postcode","billing_city","billing_region","billing_country_id","billing_telephone");
foreach($headers as $header){
	echo addThis($header);
	if($header != $headers[count($headers) - 1])
		echo ",";
}
echo "\n";
//Rows
while($qs =  mysql_fetch_assoc($query_Row))
{
	foreach($headers as $header){
		//Split Address Lines In 3 Fields (billing_street1, billing_street2, billing_street3)
		if(($header == "billing_street1")||($header == "billing_street2")||($header == "billing_street3")) {
			if($header == "billing_street1") {
				$address = $qs["billing_street"];
		   		$addressLines = preg_split('/[\n\r]+/', str_replace(array('"', '\\'), array('""', '\\\\'), $address), 3);
	   			$i=0;
				while($i<3){
					if(isset($addressLines[$i]))
						echo '"'.$addressLines[$i].'"';
					else
						echo '""';
					$i++;
					echo ",";
				}
			}
		}
		else {
			echo addThis($qs[$header]);
			//Add Comma
			if($header != $headers[count($headers) - 1])
				echo ",";
		}
	}
	echo "\n";
}
echo "</pre>";

function addThis($text){
	return '"' . $text . '"';
}
/* This is the final SQL query generated */
SELECT
e.customer_id AS customer_id, ROUND(SUM((e.base_subtotal-IFNULL(e.base_subtotal_refunded,0)-IFNULL(e.base_subtotal_canceled,0))*e.base_to_global_rate), 2) AS orders_sum_amount, ROUND(AVG((e.base_subtotal-IFNULL(e.base_subtotal_refunded,0)-IFNULL(e.base_subtotal_canceled,0))*e.base_to_global_rate),2) AS orders_avg_amount, COUNT( e.entity_id ) AS `orders_count` , `_table_email`.`email`, `_table_prefix`.`value` AS `prefix`, `_table_firstname`.`value` AS `firstname`, `_table_lastname`.`value` AS `lastname`, `_table_default_billing`.`value` AS `default_billing`, `_table_billing_company`.`value` AS `billing_company`, `_table_billing_street`.`value` AS `billing_street`, `_table_billing_postcode`.`value` AS `billing_postcode`, `_table_billing_city`.`value` AS `billing_city`, `_table_billing_region`.`value` AS `billing_region`, `_table_billing_country_id`.`value` AS `billing_country_id`, `_table_billing_telephone`.`value` AS `billing_telephone`
FROM `sales_order` AS `e`
INNER JOIN `customer_entity_varchar` AS `_table_prefix` ON (_table_prefix.entity_id = e.customer_id) AND (_table_prefix.attribute_id=4)
INNER JOIN `customer_entity_varchar` AS `_table_firstname` ON (_table_firstname.entity_id = e.customer_id) AND (_table_firstname.attribute_id=5)
INNER JOIN `customer_entity_varchar` AS `_table_lastname` ON (_table_lastname.entity_id = e.customer_id) AND (_table_lastname.attribute_id=7)
INNER JOIN `customer_entity_int` AS `_table_default_billing` ON (_table_default_billing.entity_id = e.customer_id) AND (_table_default_billing.attribute_id=13)
LEFT JOIN `customer_address_entity_varchar` AS `_table_billing_company` ON (_table_billing_company.entity_id = _table_default_billing.value) AND (_table_billing_company.attribute_id=22)
LEFT JOIN `customer_address_entity_text` AS `_table_billing_street` ON (_table_billing_street.entity_id = _table_default_billing.value) AND (_table_billing_street.attribute_id=23)
LEFT JOIN `customer_address_entity_varchar` AS `_table_billing_postcode` ON (_table_billing_postcode.entity_id = _table_default_billing.value) AND (_table_billing_postcode.attribute_id=28)
LEFT JOIN `customer_address_entity_varchar` AS `_table_billing_city` ON (_table_billing_city.entity_id = _table_default_billing.value) AND (_table_billing_city.attribute_id=24)
LEFT JOIN `customer_address_entity_varchar` AS `_table_billing_region` ON (_table_billing_region.entity_id = _table_default_billing.value) AND (_table_billing_region.attribute_id=26)
LEFT JOIN `customer_address_entity_varchar` AS `_table_billing_country_id` ON (_table_billing_country_id.entity_id = _table_default_billing.value) AND (_table_billing_country_id.attribute_id=25)
LEFT JOIN `customer_address_entity_varchar` AS `_table_billing_telephone` ON (_table_billing_telephone.entity_id = _table_default_billing.value) AND (_table_billing_telephone.attribute_id=29)
INNER JOIN `customer_entity` AS `_table_email` ON (_table_email.entity_id = e.customer_id) WHERE ( e.entity_type_id = '11' ) AND ( e.state != 'canceled' ) AND ( e.created_at >= '2010-06-07 00:00:00' AND e.created_at <= '2020-01-01 23:59:59' )
GROUP BY `e`.`customer_id`
ORDER BY orders_sum_amount DESC
  • 22nd, Jul 2010

PHP mysqldump Script – Backup a MySQL Database

Tags: , , , , , ,

/**
* PHP mysqldump Script
* www.unexpectedit.com
*/
$dbname="magentoDatabase";
$dbhost="localhost";
$dbuser="root";
$dbpass="123456";

$backupFile = $dbname . date("Y-m-d-H-i-s")  . '.gz';
$command = "mysqldump --opt -h $dbhost -u'$dbuser' -p'$dbpass' $dbname | gzip > $backupFile";
$returned = system($command);
echo $returned;
  • 9th, Jul 2010

Useful Queries For Magento

TAGS: None

<!-- Order Lines  -->
SELECT
 sales_order.increment_id AS "Order Number",
 e.item_id AS "Item Number",
 e.sku AS "Product Code",
 IF(e.qty_canceled > 0, 'cancelled', 'shippep') AS "Item Status",
 IF(e.qty_canceled > 0, e.updated_at, '') AS "Cancelled / Returned date",
 IF(e.qty_canceled > 0, '0', e.qty_ordered) AS "Number of Items sold",
 e.qty_canceled AS "Number of Items Returned / Cancelled",
 e.base_row_total AS "Net Value",
 e.base_tax_amount AS "VAT Value",
 e.discount_amount AS "Net Discount Value",
 ROUND((sales_order.base_shipping_amount/1.15),2) AS "NET P&P Value",
 sales_order.base_shipping_amount AS "P&P VAT"
FROM sales_flat_order_item  AS e
RIGHT JOIN sales_order ON sales_order.entity_id = e.order_id
ORDER BY sales_order.increment_id DESC
  • 7th, Jul 2010

Magento Report with Column with Fixed Text

Tags: , , , ,

    protected function _prepareCollection()
    {
        $collection = Mage::getResourceModel('customer/customer_collection')
            ->addNameToSelect()
            ->addAttributeToSelect('email')
            ->addAttributeToSelect('created_at')

            ->joinAttribute('billing_company', 'customer_address/company', 'default_billing', null, 'left')
            ->joinAttribute('billing_street', 'customer_address/street', 'default_billing', null, 'left')
            ->joinAttribute('billing_postcode', 'customer_address/postcode', 'default_billing', null, 'left')
            ->joinAttribute('billing_telephone', 'customer_address/telephone', 'default_billing', null, 'left')
            ->joinAttribute('billing_city', 'customer_address/city', 'default_billing', null, 'left')
            ->joinAttribute('billing_country_id', 'customer_address/country_id', 'default_billing', null, 'left');

		//This line will add a default text 'No' to all column
		$collection->addExpressionAttributeToSelect('rentable', "CONCAT('No')", array("email"));            

        $this->setCollection($collection);
        //echo $collection->getSelectSql();

        return parent::_prepareCollection();
	}

© 2010 unexpected[it]. All Rights Reserved.

This blog is powered by Wordpress and a modified version of Magatheme