• Welcome to ASR. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Forum Search Index Not Updating

It is in the top 10 larges files!
How much does the ASR "Call for Humor" post - with near-1800 pages of images - contribute to this ASR data over-crowding?
Would it be possible to just off-load the image depository to another server, if images are a big contributor to this shortage?

We can always use @Boris Badinov as the scapegoat for amassing such a huge depository.:p
 
The first action IMHO would be to stay informed. So after solving the immediate issue, I would install Zabbix or something similar, start monitoring basic things like disk and resources usage and then configure some e-mail alerts based on thresholds. Alerts on disk usage and basic resources to begin with, and then on certain database tables or cache/temporary files that could be harmful. Alerts can be configured at different levels of severity, etc.

https://www.zabbix.com/documentatio...eb_interface/frontend_sections/alerts/actions

If everything is in the same partition, it's a normal practice to create different partitions for different application areas, in particular for the database.

Also, sometimes there is cache data that can be erased when too large without any major loss. One can develop a script in Zabbix that is triggered when some threshold is reached that performs a cleaning action, for instance erasing some information that isn't critical and keeps the application working.

I'm sure there is a guy around here that has done this many times on many servers. I'm not one of those, but I've seen them working for years and I think that's more or less what they would recommend. First things first: have more insight on what's going on. Motorize, rise alerts, optimize.

My two cents.
 
The first action IMHO would be to stay informed. So after solving the immediate issue, I would install Zabbix or something similar, start monitoring basic things like disk and resources usage and then configure some e-mail alerts based on thresholds.
Digital Ocean has full set of alerts for system resources. What Elastic Search considered high watermark was not known to me until this latest incident. Even now, I can't override its default despite what they say in their documentation.
 
@amirm are you using this feature?
I am not as our performance has been quite good (likely due to SSD storage). I did investigate it for a while but at the end, didn't go with any of it.
 
I am not as our performance has been quite good (likely due to SSD storage). I did investigate it for a while but at the end, didn't go with any of it.

Given everything you have shared so far I think throwing more storage at the problem is the best option. I've developed a lot of databases over the years, and if the schema was well written to start with, more hardware is the only option.
 
I have done all of that (spent a day on it). Nothing seems to stop it from refusing to index due to watermark levels. Of course it is possible what I did was not correct. It doesn't matter that much anyway as we are 92% full right now so we need to move over. Hate to have the index fill up the space and stop the rest of the forum from working.
 
So your plan is to increase space to ensure the whole application is working and then fix the index? Sounds good to me given the situation. As I understand it, the index can be rebuild at any time, so even deleting it could be an option and then performing delete and ingest again, if necessary.
 
Correct. We also need to change OS as our current one is EOL. So I had to provision a new server. Otherwise I could have just increased the storage on the current one.

And yes, there is a method to get it to rebuild the indices.
 
Sounds challenging but completely sound. I've been in many migrations and from experience I know it's hard to perform those operations alone, with many small decisions involved. I'm not a sys admin but I've supervised, planned and coordinated many similar operations. I know you have got a lot of experience but even so, If you want to share anything about the process to get a second opinion or just a "digital hug" here I am (and others of course). Good luck out there!
 
Nuke it all.

Start over.

(except for your Test Reviews)

We'll fill it back up in a few days anyway.
 
Oh, and by the way. I would encourage you to create different partitions in the new server for different applications or services. For instance, if the Elastic has it's own partition, it can't possible affect the rest of the site if it fills up. Another one for the database it's a common practice too, following the same reasoning. Also, it's very advisable to leave 20% or 30% of the new space unassigned to any partition, so you can use it at any moment to increase the space of any partition in case of urgency, and gain time. It's difficult to predict what will grow faster.
 
Oh, and by the way. I would encourage you to create different partitions in the new server for different applications or services. For instance, if the Elastic has it's own partition, it can't possible affect the rest of the site if it fills up. Another one for the database it's a common practice too, following the same reasoning. Also, it's very advisable to leave 20% or 30% of the new space unassigned to any partition, so you can use it at any moment to increase the space of any partition in case of urgency, and gain time. It's difficult to predict what will grow faster.
One loses storage efficiency with that kind of partitioning. Elastic's watermarks is supposed to kind of do that without losing that efficiency due to external fragmentation.
 
One loses storage efficiency with that kind of partitioning. Elastic's watermarks is supposed to kind of do that without losing that efficiency due to external fragmentation.
Not sure If we are talking about the same concept. In Linux the term partition is unambiguous but I think in windows they are most frequently called volumes. What I mean is to divide the space available between several virtual disks, which from the point of view of the applications are isolated drive units. I use the term partition as I guess what you get in the VPS is a single disk unit, which then could be partitioned, like a physical disk can be in any OS.

In that scenario, the Elastic wouldn't see the other ones, it would had its own disk.

Because each application has its own disk (volume), no application can take out space from another one, so it's a way of isolating the problems. For example, this is why in Linux servers there is always a partition for the boot, to ensure that even if all disks get full, the system can boot with basic capabilities. Or ff the watermark of the Elastic is reached, you know that isn't going to affect the database. Aligning the physical structure of the partitions (really virtual but physical from the point of view of the applications) with the operational needs is a rather common practice among sys admins. I haven't seen any server (I've only worked with Linux servers) where all important components didn't have their own disk (partition, volume). That way, each disk has a particular needs of backup, performance, etc. aligned with the actual need of the application. For instance, a database could benefit from a type of disk and certain tuning, so at any point you can add a different disk and move the entire database to the new one just changing the mount point (this is Linux jargon). It also facilitates backups and alarms. For instance, if you have and alarm on the database partition, it's easier to interpret immediately than an alarm on a disk that contain all the applications. In many managed servers like VPSs you can have backups but they are from whole disks, hence again, if you have a partition for the database you can copy the whole unit which is far more easy than selecting a bunch of folders.

Finally, the reasoning behind leaving some space unassigned is simply that enlarging a disk partition is a trivial and risk free operation in any OS, whereas shrinking one is a rather costly and potentially troublesome one. So there is always a tendency to be conservative about the space given to each disk (partition) and then watch each one and give more space individually as needed. In the contrary, if you

I understand that in managing a single server with limited time and all other constraints and circumstances is very tempting to have a single disk for everything. It can work certainly, but it has a tendency to be easy at first and then difficult to maintain and grow later. Sys admins tend to focus on minimizing the difficulty of maintenance and invest a lot of time at the beginning.

Of course, this all comes from experience with Linux servers, with Windows I have no experience at all at this level, so perhaps the "tradition" is different there.

In the end the most important thing is to understand the consequences of the two approaches, single disk unit vs multiple ones, and commit to one of them.

Again, good luck!
 
Overnight, I noticed that ASR website was under some (type of) maintenance. As refreshing (F5) the ASR homepage, gave me a maintenance prompt...
Was this "maintenance" something routine or related to site' indexing issue(s)?
Hoping the problem has been resolved... not that I have noticed any forum problems, when using Search.
Again, good luck!
+1
 
Back
Top Bottom