Hosting for a social network - what should it be like? VKontakte: how the social network works Hosting power for a new social network.

Dreams of big income and serious earnings should not remain only in thoughts; you need to do something to achieve success. There are quite a few options, and one of them is launching your own website. A mini social network is a great idea, because this niche is now actively developing and there are many fans of thematic resources.

A miniature social network has one big advantage: after its creation, you won’t have to compete with such giants as VKontakte or Facebook. Of course, it would be nice to collect people from these sites, but you will have a defined target audience, and this is how you will attract attention.

Social network is a great source of profit

There are so many websites that have not been created in the social niche. networks. There are niche networks for doctors, psychologists, children, and so on. They are incomparable to, however, their owners also make a profit. Naturally, it is unlikely that you will be able to achieve the same income, but even a tenth is enough to become a wealthy person.

Imagine that your website gets into at least the TOP 50 popular social networks, and you will receive 50 times less money. According to the latest data, VKontakte earned $1.5 billion in a year from targeted advertising alone. 50 times less – that’s $30,000 ($2500 per month). Agree, this is also a good profit, especially since this is only income from advertising.

It takes several hours to create a blog, but a mini network takes much longer to create. This is due to the fact that there are many more functions there. You can read a separate article to get an idea of ​​this process. The site must be perfect from all sides, so serious efforts will have to be made.

The only option to make the development of such a resource easier is to save money and order it from professionals. Web studios and private webmasters will help with this. Search for professionals on Weblancer, study reviews, see examples of work and choose the right candidate. Also, do not forget that later you will have to invest in website promotion.

Mini social network should be interesting

After launching such sites, you have to wait a long time for natural traffic to arrive. A lot of money is spent on advertising, and for it to be effective, you will also have to understand it. Read so as not to waste money and bring as many visitors as possible.

Various features and combinations will help make the project interesting. Give free rein to your imagination, and first of all come up with an unusual theme. Perhaps it will be some kind of sport, a mini social network for schoolchildren or elderly people, cooking lovers, fans of some game, and so on.

Alternatively, the social network can be combined with other types of sites. Embed a blog, add some useful service, add online advertising functions, earning money, and so on. In general, you need to come up with a feature that no other project has in order to capture as much attention as possible.

It will take a lot of effort to create your own, even a small social network. Many people try to do this, but abandon projects halfway through, because they can no longer work without return, and this is exactly what any business requires. In conclusion, I would like to advise you to read about it, it will be useful to you at the initial stage.

The server is a necessary element in the process of creating serious websites or . In the case of the latter, server capacity generally plays, perhaps, a decisive role, since users of social networks constantly watch high-definition videos, use applications, and so on.

Alas, you won’t be able to save money on creating a server room, and it’s hardly even worth trying - the result can be very, very sad. It always makes sense to create server capacity with a reserve, because you probably hope and expect that your project will grow, otherwise why create a website at all?

For stable operation of a large resource or a dynamically developing social network, good server capacity is needed. Alas, it is impossible to provide exact data on the required power in one article, since a lot depends on how the site is made, how well it is optimized, etc.

There are three main criteria for server power - CPU frequency, amount of RAM And disk subsystem performance. In principle, a server priced at $400-600 is sufficient for a small site, but serious projects require several servers of significantly greater capacity. If you don’t save money and give preference to proven and well-established brands - such as Dell, MSI or HP - you should be prepared to pay from one and a half to two and a half thousand dollars for each server.

Of course, you can choose a much more budget option with similar characteristics - you can even buy used servers that are in excellent condition, since there are quite a lot of them on the market. But, in any case, good server capacity costs good money. Let's say, for 20 thousand unique visits per day, a computer with an Intel Core Quad 9550 processor (12MB cache), 6GB of memory and two WD 7200 RPM drives will probably be enough. However large social networks attract significantly more users, so it’s quite obvious that one server is not enough here. In general, to select the right servers, consultation with specialists is necessary - there are many types of servers for different purposes, and it would be wiser to purchase computers in accordance with the type and characteristics of your resource.

Buying servers is an important step, but setting them up is a much more serious and delicate matter. High-quality server setup is only possible in cooperation with competent and experienced specialists, which means that you won’t be able to save money here either. In general, the creation of any serious website or ambitious social network involves very decent expenses, and those who are looking for something cheaper should pay attention to other projects.

The matter is not limited to setting up servers. The stability of the servers is the key to the successful operation of the resource, so servers require constant technical support. This, of course, is an important expense item, but the good condition of the servers should always come first, because in the event of any problem, the operation of the site may be interrupted indefinitely, which will have an extremely negative impact on traffic and monetization. The server is the heart of any Internet project, so the attitude towards it should be appropriate.

Almost 100 million users - this is the VKontakte audience that
need to be serviced. Fast and without interruption. For a long time the details of the technical
the implementation of VKontakte remained a secret. But recently the most popular in Russia
The social network shed some light on how it actually works. At the end
October the HighLoad++ conference was held in Moscow, at which representatives
VKontakte, represented by Pavel Durov and Oleg Illarionov, finally told something
about the architecture of a social network.

The guys were literally inundated with questions on completely different aspects of their work.
VKontakte, including technical ones. Still would. It is easy to imagine the load on
server part of the service: how many people do you know who do not use this
social network? How much time do you spend there, wasting precious hours of your
life to communicate with friends, watch videos, games, music? Math is pretty
is simple: a fabulous number of users * a lot of time spent on
resource = prohibitive number of requests to web servers and database +
Terabytes of constantly downloaded and viewed photos, videos and audio.

VKontakte statistics

• 99.5 million accounts.
• 40 million active users worldwide (comparable to
  Internet audience in Russia).
• 11 billion requests per day.
• 200 million private messages per day.
• The video stream reaches 160 Gbit/s.
• More than 10 thousand servers, of which only 32 are frontends on nginx
  (the number of servers with Apache is unknown).
• 30-40 developers, 2 designers, 5 system administrators, many
  people in data centers.
• About 10 hard drives fail every day.

Platform

For us, of course, the architecture of the project is of particular interest: how
the main components of the system interact, which are our own developments
required, what tricks had to be used. But before we move on to
it, you need to familiarize yourself with the basic things - the technologies used and
products.

Debian Linux is used as the main operating system - a solution
time-tested, one of the oldest and most stable modern
distributions. Used to balance the load between application servers
Nginx HTTP server running in reverse proxy mode. His responsibilities include
maintain a connection with the user’s browser and transmit requests to servers,
responsible for the execution of PHP code, and also control the hit
the result back to the browser. PHP code is executed using the mod_php module for
Apache - there are quite a few alternatives, especially protocol-based ones
FastCGI, but VKontakte management took a more conservative path in this
issue using the most time-tested solution. No special
PHP code performance optimization systems are not used (for example, in
Facebook wrote their own compiler from PHP to C called HipHop), the only
external optimization is op-code caching using a publicly accessible
XCache solutions.

The situation with data storage looks quite vague: on the one hand,
We actively use our own database management system written in
C and created by the “best minds” of Russia, on the other hand, MySQL was often mentioned in the role
main storage. I’ll tell you more about my own VKontakte database
below. Speaking about data storage, one cannot fail to mention such an important aspect as
caching frequently used information (its location in RAM
for quick access). A very popular product in this industry is used for this.
areas - memcached. If you haven't heard: this system allows you to carry out very
simple atomic operations such as locating and retrieving arbitrary
key data. The main feature is lightning fast access and
the ability to easily combine RAM of a large number of servers
into a common array for temporary storage of “hot” data.

Third-party projects that are not key to VKontakte are often implemented
either using rather exotic solutions, or, conversely, using the most
simple technologies. For example, an instant messaging service is implemented
on node.js (you can read more about this development in the article "" in) using the XMPP aka Jabber protocol (we'll get to that
we'll be back). Video conversion is implemented using the simplest and most effective
library - ffmpeg, it also runs the very popular video player VLC.

Architecture

The most noticeable difference from the architecture of many other large
Internet projects is the fact that VKontakte servers are multifunctional.
Those. there is no clear division into database servers, file servers, etc. - They
simultaneously used in several roles. At the same time, redistribution
roles occurs in a semi-automatic mode with the participation of system
administrators. On the one hand, this optimizes the efficiency of use
system resources, which is good, but on the other hand, it increases the likelihood of conflicts on
operating system level within one server, which entails
stability problems. However, despite the use of servers in different
roles, the project's computing power is typically used at less than 20%.

Load balancing between servers occurs according to a multi-level scheme,
which includes balancing at the DNS level (the domain is served with
using 32 IP addresses), as well as routing requests within the system, and
different servers are used for different types of requests. For example, generation
pages with news (now it is commonly called a microblog) works in a cunning way
scheme using the parallel sending capabilities of the memcached protocol
requests for data on a large number of keys. In case of absence
data in the cache, a similar request is sent to the data storage system, and
the results obtained are sorted, filtered and discarded
already at the PHP code level. This functionality works in a similar way on Facebook.
(they recently exchanged experience), only instead of their own DBMS on Facebook
use MySQL.

A large amount of software has been developed within the walls of VKontakte, which is more
precisely meets the needs of the project than available opensource and
commercial solutions. In addition to the mentioned own DBMS, they have a system
monitoring with notification via SMS (Pavel himself helped layout the interface),
automatic code testing system and statistics and log analyzers.

The project uses fairly powerful equipment, approximately
The following server characteristics are named:

• 8-core Intel processors (two per server, apparently);
• 64 GB of RAM;
• 8 hard drives;
• RAID is not used (replication and backup are carried out
  at the software level).

It is noteworthy that the servers are not branded, but are assembled by a specialized
Russian company. Currently, the project equipment is located in 4 data centers in
St. Petersburg and Moscow, with the entire main database located in
St. Petersburg data center, and in Moscow only audio and video are hosted. In the plans
make database replication with another data center in the Leningrad region, and
also use Content Delivery Network to increase download speed
media content in the regions.

Many projects faced with a large number of photographs often
invent their own solutions for storing and delivering them to users. About it
was the first question asked to Pavel from the audience: “How do you store images?” - "On
disks!" One way or another, representatives of VKontakte said that this whole bunch
photographs of all colors and sizes are simply stored and transferred from the file system
(use xfs) a large number of servers, without additional frills.
The only thing that confuses us is the fact that other large projects do not have this approach.
worked - they probably didn't know the magic word :).

That very own database in C seems no less magical.
This product, perhaps, received the main attention of the audience, but at the same time
almost no details about what it actually is,
was never made public. It is known that the DBMS was developed by "the best minds"
Russia, winners of Olympiads and TopCoder competitions, and also that it is used
in the most heavily loaded VKontakte services:

• Private messages
• Messages on the walls
• Statuses
• Search
• Privacy
• Friends lists

Unlike MySQL, a non-relational data model is used, and most
operations are carried out in RAM. The access interface represents
is an extended memcached protocol. Specially composed keys
return the results of complex queries (most often specific to a particular
service).

The system was designed taking into account the possibility of clustering and automatic
data replication. The developers would like to make this system
universal DBMS and publish it under GPL, but it is not yet possible due to the high cost
degree of integration with other services.

• The development process is close to the Agile methodology with weekly
  iterations (cycles), within which all stages of development take place:
  planning, requirements analysis, design, development and
  testing.
• The operating system kernel has been modified (to work with
  memory), there is its own package base for Debian.
• Photos are uploaded to two hard drives of one server
  simultaneously, after which a backup copy is created on another server.
• There are many improvements to memcached, incl. for more stable and
  long placement of objects in memory; there is even a version
  ensuring data safety.
• Photos are not deleted to minimize fragmentation.
• Decisions on the development of the project are made by Pavel Durov and Andrey Rogozov,
  Responsibility for services lies with them and with the developer who implemented it.
• Pavel Durov has been saving money for hosting since his 1st year :).

Subprojects

Audio and video services are secondary to the social network; they
the creators of the project are not particularly focused. This is mainly due to the fact that they
rarely correlate with the main purpose of using a social network - communication, and
also create a lot of problems. Video traffic is the main expense item
project, plus well-known problems with illegal content and claims
copyright holders. 1000-1500 servers are used for video transcoding, on
It is also where it is stored. Media files are banned by hash when deleted upon request
copyright holders, but this is ineffective and it is planned to improve this
mechanism. Obviously, we are talking about developing a more intelligent algorithm
recognition of audio and video content by tags, as, for example, this is implemented
on YouTube, where an uploaded video that violates the license may be
automatically deleted within a few minutes after downloading.

As you know, some time ago it became possible to communicate on
VKontakte via the Jabber protocol (aka XMPP). The protocol is completely open and
There are a lot of opensource implementations. For a number of reasons (including problems
integration with other VKontakte services) it was decided to create in a month
own server, which is a layer between internal services
VKontakte and the implementation of the XMPP protocol. It is implemented in node.js - choice
due to the fact that almost all project developers know JavaScript, and
In addition, this is a good set of tools for implementing the task. Difficult moment
began working with large contact lists. Many users have a number
VKontakte friends are measured in hundreds and thousands, the shift activity is high
statuses: people appear and disappear online more often than in other similar
situations. In addition, it was necessary to implement close integration with internal
personal messaging system VKontakte. As a result, the service is 60-80
thousand people online, at its peak - 150 thousand. TCP/HTTP load balancer
HAProxy processes incoming connections and is used for distribution
queries on servers, as well as deployment of new versions.

When choosing a data storage system, we thought about non-relational storage systems
data (in particular, about MongoDB), but in the end they decided to use the usual
MySQL. The service operates on 5 servers of different configurations, on each
which code runs on node.js (4 processes per server), and on the three most
powerful - also MySQL. An interesting feature is the lack of connection between
groups of friends in XMPP with groups of friends on the site - done by request
users who did not want their friends to see over their shoulders what
group they are.

46.1K

You want to know, how to create a social network? Like any website, it requires the use of the right software, thoughtful planning and implementation. Today I am going to help you and explain how to create a social network.

Explain your idea and concept

Before you start working, you need to make sure that your idea is worth something. Ask yourself some questions to understand what this idea involves.

What niche am I going to occupy?

This is the most important question you should ask yourself before creating a social network. Trying to implement something like Facebook will not succeed. But perhaps there is a specific niche that you are passionate about filling and that could bring benefits:

I'm assuming you already have an idea, but it's important to make sure your expectations are realistic. Don't expect your resource to easily become the next Facebook or My Space.

Make sure you have a clear idea of ​​the niche or specific topic for your social network. Maybe it will be for musicians or animal lovers.

Are there any social networks that already fill your niche?

As with any good business, it's important to do your research and figure out who you'll be competing with. Are there already social networks that fill the niche you are looking to fill? If so, what are they doing right? What would you do better?

Before you create a social network, take notes about what you like and don't like. This way you'll know what you need to do to get an edge.

What services or features do I want to offer my audience?

This is partly explained in the question above, but I want to make sure you actually think about it seriously. What are you going to offer your audience that will make them want to be a part of your community?

If you're thinking about how to build a social network from scratch for animal lovers, you might consider partnering with pet supply companies to offer some kind of discount to your audience. Or perhaps you'll offer specialized services, such as pet training or videos on the topic.

Think carefully and make sure that the solution you choose will be of interest to your audience.

How much do I want to spend?

It's obvious that everyone's goal is to spend less money to earn a higher income, but this is not always realistic. Social networks (like any website) require financial investments. They can range from several hundred to several thousand dollars. Think about how much you are willing to invest to make things happen.

For those who don't know where to start, here are some typical website costs, ( Please note that these are conservative estimates.).

Hosting

This only applies if you decide to use purchased software that will require installation on your own site. You'll likely want to launch your new site on VPS hosting to gain additional control.

Of course, there are cheaper options, but future package updates may cause glitches. I would recommend starting with a VPS, but you can choose whatever option suits you best. Cost – $30 – 100.

Social Network Software

Before you create a social network yourself, you need to select the appropriate software. This can be ready-made software that includes almost everything you might need. Or online designers that offer many options. These are generally the best choice because they eliminate the huge costs associated with creating a resource from scratch. Cost – from $99.

Individual design

Perhaps, after studying the market, you will find that existing software products do not meet your requirements, and therefore decide to create your own CMS for a social network. Don't expect to get anything that works reasonably well for less than a few thousand dollars. Filling with content, typical design and development of a blog for a good social network costs about $3,000 – 6,000.

Promotion

You will need to purchase your own domain. I recommend checking out the likes of Upwork.com and Fiverr.com for low-cost promotion options. Cost – from $100.

SEO (Search Engine Optimization)

Search Engine Optimization is critical to getting traffic to your website. But there are too many scammers in this field, and you can spend thousands of dollars trying to find someone with the right skills. Therefore, it is more profitable to invest in your own knowledge. Cost - $300 per month through SEO Book or more than a thousand through other SEO companies.

Do I want to do the work myself or will someone do it for me? Who will carry it out?

What is the expected load? How many people can potentially visit the social network being created? This is important in order to choose the right software.

Of course, there are still questions about what it takes to create a social network. But these are the main ones. I would recommend briefly jotting down your answers.

Use existing software or create your own?

The next step is to research what ready-made solutions are available on the market. There are many options: place it yourself ( you will be responsible for raising the website and maintaining it in good condition) or placement with a service provider ( you pay a subscription fee and they monitor the site).

To make a decision, it is important to understand whether there are options available that suit your needs or whether you need to consider developing a website from scratch. Keep in mind that custom development is expensive, so make sure you discount all available options before going down this route.

Let's look at some of the available solutions. This will allow you to understand how much it costs to create a social network.

PHPFox

PHPFox has been around for years and is one of the least expensive options. There will always be people who complain about every product released, but the fact remains that there are a lot of cool social networks running on PHPFox.

The cost is reasonable, the investment has dropped quite a bit since the main company rebranded to Moxi9 and now you can purchase PHPfox for just $99:

I believe that making a smart investment saves you from having to do everything yourself. For less than a hundred dollars, you get the basics to get started, and if your site does well, you can consider other options. Such as creating a template for a custom social network. But after you start making profit from the social network.

You should not discount such options, since they can quickly solve the problem with minimal investment. You don't want to spend thousands of dollars only to find out your idea flops, so start by minimizing costs and keeping costs low.

Social Engine

Social Engine is considered one of the best platforms for creating a social network along with PHPFox. Their offerings include a cloud version ( you can use their product on your servers without the need for self-installation), which will cost you $29 per month. And also a version of the CMS, which will be entirely yours, and will cost $299:

Personally, I think the cloud solution is a little better since you don't have to deal with setting up hosting. The downside is that you lose the ability to customize it yourself and don't have complete ownership of the product. Either way, the SE is a great option to consider.

Dolphin Smart Community Builder

The platform offers a lot of functionality at a reasonable price. Although it will cost you more than PHPFox, you may find that all solutions better suit your needs:

How much does it cost to create a social network: Dolphin prices range from $399 to $799. It's certainly more expensive than most options, but it's much cheaper than developing from scratch. So don't discount this fact.

Evoq Social

If the social network you want to create is more business style, then you should definitely look at Evoq Social. This solution from DNN Software will help you create a truly amazing social network.

Anyone who is planning to create their own own social network(assuming based on BuddyPress), should be equally concerned about both the quality of services provided to end users and their own capacity for continuous access to the site. Moreover, often the capabilities of the server (and I’m talking about them) determine the growth potential of your network.

You all understand that regular WordPress itself is quite power hungry (12-17 MB with the usual set of plugins - 10 pieces, no more). WordPress MU, due to its support for multiblogs, consumes even more (by a couple of megabytes). BuddyPress adds additional megabytes to WPMU, which often leads to an error about insufficient allocated resources to activate the necessary plugins. As a result, I often came across the following:

1. WordPress MU;
2. BuddyPress;
3. bbPress forum;
4. 5-10 plugins

become the limit for the site - nothing else can be activated due to the lack of the same php_value memory_limit. And what’s most interesting is that even such idiots as masterhost, nic, mirohost and some others (both Russian and Ukrainian [I don’t recommend hostpro.ua]). Naturally, the problem can be solved quite easily (but this also means that it is quite expensive) - switching to another hosting plan.

But I have a question: is it advisable to use the limited resources of virtual servers for such serious projects as global/regional social network? Virtual hosting is only good if the website is in its infancy. After passing a certain stage (on WPMU+BuddyPress this is approximately 500 uniques per day and 3-4 times more than usual visits) you Not a single virtual plan of any hoster is enough. From this point on, you will have access and performance problems. To solve them, you will sooner or later begin the process of transition or VDS/VPS, or to purchase/rent your own server. Transferring all data will be associated with huge costs (mainly psychological and loss of time), since nothing can be lost and at the same time it is highly undesirable to close the site for technical work so as not to frighten off visitors.

Brief information on VDS/VPS

VPS (virtual private server) - virtual private server. VDS (Virtual Dedicated Server) is a virtual dedicated server, which is actually synonymous with VPS, although it has differences in the organization of resource sharing.

You have at your disposal personal fully functional server, when working with which you will feel as if you own a separate computer. You will be able to log into the VPS with administrator rights (root) and fully manage its software: configure, reinstall, delete, reinstall libraries. Agree that the virtual server did not give you such freedom.

There are, of course, some disadvantages that lie in the VPS and VDS technology itself. After all, VPS is only the illusion of a separate server, created by a special program installed on the hoster’s computer. This program allows you to divide computer resources and provide each individual owner of a virtual dedicated server with his own limit of processor time, disk space, RAM, etc. As a result, multiple people owning a VDS share the hardware of the same computer while remaining software independent and isolated from each other in terms of sharing resources.

Therefore, my conclusion is to immediately use VDS / VPS, since your own server is still too expensive.

Let's look at the characteristics that a decent hoster should have.

1. The first and perhaps most important thing is uptime, that is, the availability of your hoster’s servers. This value, in my opinion, should be higher than 99.9% for completely reliable operation (here is an example so that you can better understand: out of 100 days of continuous operation of the site, only 12 hours were it unavailable, this means that uptime = (100*24 - 12 )/100*24 = 0.995 or 99.5%). The absence of an uptime indication on the hosting company’s website may suggest that it is not as high as we would like.
2. Flexibility of settings. You have to decide what you need, and the first condition for social networks with large daily visits is productivity. For serious projects, you should not skimp on resources. That is why it is not you who should adapt to the capabilities of the hoster, but the host itself should offer you a fairly wide choice.
3. Possibility of simple upgrade current configurations, that is, upon reaching the RAM limit or processor core load, you should be able to purchase additional parameters without complications or delays.
4. Safety. I think everything is clear. Nobody wants your site to get hacked or you to be dependent on anyone else.
5. Customer support. An important indicator of the quality of the hosting provided is the ability to always get an answer to your question.

There are many more parameters by which you can evaluate different hosters, but these, in my opinion, are the main ones. I have come across many plans and tariffs, so I have a certain understanding of the market. I would not recommend using the hosters that I have already named for a number of reasons:

1. they have been on the market for a long time, but at the same time this is also their problem - there are an incredibly large number of clients, which reduces the time allocated to each client (the staff is not unlimited);
2. high cost due to the fact that they already have a brand for which you overpay;
3. lack of flexibility when working with clients - due to their scale. Bigness always breeds clumsiness.

That's why I want to offer you a hosting company Avi Host, which has all the characteristics mentioned above and is devoid of many of the disadvantages of masterhost, nic, mirohost and others. Avi Host specializes in VDS / VPS hosting, which is great for creating a social network based on BuddyPress due to its flexibility, reliability, performance and, what is not less important, low prices. You can configure your server, having determined which parameters are most important to you, read about advantages of this company, get quality advice, and also familiarize yourself with the list dedicated servers located in different countries of the world.